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ABSTRACT 

This  report  presents  the  final  results  of  a  study  on  the  feasibility  and  usefulness  of  data 
fusion  for  the  CP- 140  Aurora  maritime  patrol  aircraft.  Relevant  sensor  fusion  concepts 
and  terminology  have  been  defined  along  with  a  description  of  the  CP- 140  operational 
environment  and  information  sources.  An  analysis  of  applicable  sensor  fusion  processes  is 
presented  followed  by  a  discussion  on  the  expected  performance  improvements.  Finally,  a 
three  step  incremental  approach  is  proposed  with  recoverable  steps  where  different  level 
of  fusion  sophistication  can  be  implemented  based  on  the  availability  of  the  technology  and 
the  actual  status  of  the  sensors  on  the  aircraft. 


RESUME 

Ce  document  presente  les  resultats  d'une  etude  de  faisabilite  visant  a  porter  la  fusion  de 
don  nee  s  k  bord  de  l'avion  de  surveillance  maritime  Aurora  CP- 140.  En  premier  lieu,  les 
concepts  et  la  terminologie  relatifs  a  la  fusion  de  donnees  sont  clairement  d^finis  ainsi  que 
fenvironnement  operationnel  du  CP- 140  et  les  sources  d' information  disponibles.  Une 
analyse  de  certains  concepts  de  fusion  de  donnees  applicables  au  CP-140  est  ensuite 
presentee  suivie  d'une  breve  discussion  sur  les  eventuels  benefices.  Finalement,  une 
approche  en  trois  6 tapes  est  proposee  pour  1'implantation  des  niveaux  de  sophistication  de 
fusion.  Ces  etapes  dependent  grandement  de  l'etat  actuel  des  capteurs  du  CP- 140  et  de  la 
disponibilite  de  la  technologie  sur  la  fusion. 


P228813.PDF  [Page:  6  of  90] 


P228813.PDF  [Page:  7  of  90] 


UNCLASSIFIED 

iii 


TABLE  OF  CONTENTS 


ABSTRACT  /  r£SUM£ 

EXECUTIVE  SUMMARY 
LIST  OF  ACRONYMS 

1.0  INTRODUCTION 

2.0  SENSOR  DATA  FUSION  CONCEPTS  AND  TERMINOLOGY 

2. 1  Overall  MSDF  System 

2.2  Environment 

2.3  Energy  and  Signal  Processing  (Detection  and  Measurement) 

2.4  Sensor-Level  Data  Processing  (Tracking  and  Identification) 

2.5  Data  Fusion  Definition 

2.6  Data  Fusion  Hierarchy 

2.7  Multi-Sensor  Data  Fusion  Architectures 

2.8  Sensor  Management 

2.9  Limitation  Factors 

2.10  Automation  Issues 

3.0  DEFINITION  OF  THE  CP- 140  OPERATIONAL  ENVIRONMENT 

3.1  Mission  Requirements 

3.2  Tactical  Scenarios 

4.0  CP- 140  INFORMATION  SOURCES 

4.1  Review  of  the  Current  CP- 140  Sensor  Suite 

4.2  Review  of  the  Sensor  Upgrades  Currently  Anticipated  for  the  CP- 140 


P228813.PDF  [Page:  8  of  90] 


UNCLASSIFIED 

iv 


4.3  Review  of  the  Other  Information  Sources  33 

5.0  SENSOR  DATA  FUSION  PROCESS  FOR  THE  CP- 1 40  35 

5. 1  Top-Level  Functional  Decomposition  of  the  SDF  Process  35 

5.2  AWSDF  Tracking  and  Identification  Algorithms  for  the  CP-140  38 

5.3  Hardware  Considerations  47 

6.0  EXPECTED  PERFORMANCE  IMPROVEMENTS  50 

6. 1  Impact  on  Operator  Efficiency  50 

6.2  Impact  on  Mission  Effectiveness  52 

7.0  RECOMMENDED  IMPLEMENTATION  PLAN  55 

7.1  STEP  1:  Baseline  ATDTI  with  Radar,  ESM,  Link  and  IFF  56 

7.2  STEP  2:  Advanced  ATDTI  with  the  S  AR  and  the  FLIR  62 

7.3  STEP  3:  Full  SDF  System  Implementation  62 

7.4  Final  Considerations  on  the  Recommended  Implementation  Plan  67 

8.0  CONCLUSION  69 

9.0  REFERENCES  71 

FIGURES  1  to  12 


TABLES  I  to  XIII 


P228813.PDF  [Page:  9  of  90] 


UNCLASSIFIED 

v 


EXECUTIVE  SUMMARY 

The  CP- 140  aircraft  will  undergo  a  Life  Extension  (LE)  program  that  could  replace 
a  number  of  sensors  as  well  as  the  General  Purpose  Digital  Computer  that  provides 
command,  control,  and  data  management  of  the  sensors.  The  CP- 140  Statement  of 
Requirement  (SOR)  has  identified  “data  fusion”  as  an  essential  requirement  for  the  CP- 
140.  This  report  presents  the  results  of  a  feasibility  study  to  implement  data  fusion  aboard 
the  CP-140. 

More  precisely,  this  document  describes  how  to  provide  automatic  target  tracking 
and  identification  through  Multi-Sensor  Data  Fusion  onboard  the  CP- 140.  Relevant 
sensor  fusion  concepts  and  terminology  are  defined  along  with  a  description  of  the  CP- 140 
operational  environment  and  information  sources.  As  a  result  of  the  analysis  of  applicable 
sensor  fusion  processes,  a  hybrid  sensor  data  fusion  architecture  is  proposed  for  the  CP- 
140. 


For  the  tracking  aspect,  the  radar,  the  IFF,  and  the  FLIR  all  have  good  resolution 
and  accuracy  and  hence  are  ideal  for  central -level  fusion.  The  ESM  sensor  has  very  poor 
resolution  and  accuracy  in  the  one  dimension  (bearing)  it  measures.  Therefore  the  sensor- 
level  fusion  approach  is  selected  in  this  case  in  order  to  allow  an  ESM  track  to  first 
develop  so  that  better  data  is  available  for  the  subsequent  correlation  process.  The 
remaining  sensors,  Link  1 1  and  acoustic  sensors  are  suitable  for  sensor-level  fusion  since 
their  outputs  are  already  in  track  form  and  not  in  contact  form. 

For  the  identification  aspect,  a  central-level  fusion  architecture  is  recommended  to 
ultimately  declare  an  ID  from  the  fusion  of  the  raw  target  features  extracted  by  the  SAR 
and  FLIR  sensors.  For  the  identity  information  provided  by  the  other  sensors  (ESM,  IFF, 
Acoustics)  or  by  non-organic  sources  (Link  1 1 )  already  offering  identity  declarations  as 
their  output  data,  the  sensor-level  architecture  is  an  appropriate  approach.  This 
architecture  can  be  adapted  to  include  identity  declarations  inferred  by  operators  as  well  as 
the  ones  deduced  by  non-organic  systems. 

Since  the  types  of  data  processed  and  the  time  scales  involved  are  so  different 
between  the  under  water  sensors  and  the  above  water  sensors,  two  distinct  data  fusion 
centres  are  recommended  with  a  common  track  data  base  that  insures  the  required 
communication  between  the  fusion  centres.  Finally,  a  three  step  incremental  approach  is 
proposed  for  the  implementation  with  recoverable  steps  where  different  level  of  fusion 
sophistication  can  be  implemented  based  on  the  availability  of  the  technology  and  the 
actual  status  of  the  sensors  on  the  aircraft.  The  results  presented  in  this  report  will  be  used 
to  derive  reasonable  and  prioritized  requirements  for  the  CP- 140.  The  report  will  be 
provided  as  reference  documentation  to  the  CP- 140  Life  Extension  definition  contractor 
as  supporting  data.  The  definition  contractor  will  then  make  its  proposal  for 
implementation. 
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1.0  INTRODUCTION 

The  CP- 140  aircraft  will  undergo  a  Life  Extension  (LE)  program  that  could  replace 
a  number  of  sensors.  The  project  could  also  replace  the  General  Purpose  Digital  Computer 
(GPDC)  that  provides  command,  control,  and  data  management  of  the  sensors.  Given 
these  considerations,  Multi-Sensor  Data  Fusion  (MSDF)  has  been  identified  as  a 
technology  that  could  greatly  enhance  the  tactical  crew  abilities  to  perform  their  missions. 
The  Data  Fusion  and  Resource  Management  Group  at  Defence  Research  Establishment 
Valcartier  (DREV)  has  been  involved  for  many  years  in  the  analysis,  development, 
implementation  and  evaluation  of  MSDF  algorithms  and  techniques  for  the  automation  of 
the  target  tracking  and  identification  processes  on  a  Canadian  Patrol  Frigate  (CPF)  ship. 
Much  work  in  MSDF  has  also  been  carried  out  for  the  CPF  program  by  Unisys  GSG  (now 
Loral  Canada)  (most  of  the  time  contracted  by  DREV,  or  by  DND  with  DREV  as 
scientific  advisor)  and  there  is  a  possibility  to  use  some  of  that  technology  for  the  CP- 140 
aircraft.  Another  potential  candidate  for  data  fusion  technology  expertise  resides  at 
Westinghouse  Norden  Systems  (Melville,  NY)  who  developed  a  family  of  Integrated 
Automatic  Detection  and  Tracking  (IADT)  systems  that  are  currrently  deployed  on 
cruisers,  destroyers,  frigates,  and  amphibious  assault  ships  in  the  US  and  foreign  navies. 

In  that  context,  DREV  has  been  tasked  by  DGAEM/DMAEM  with  PMO  Aurora  as 
Project  Officer  to  investigate  the  feasibility  of  migrating  the  shipboard  MSDF  technology 
discussed  above  to  the  airborne  domain.  A  contract  was  awarded  to  Loral  Canada  to 
study  the  possibility  of  using  some  of  the  data  fusion  technology  developed  for  the  CPF  in 
the  CP- 140  context.  A  similar  contract  was  awarded  to  Westinghouse  Norden  Systems  to 
study  the  applicability  of  the  IADT  systems  developed  for  the  US  and  foreign  navies  to  an 
airborne  platform.  Following  these  efforts,  DREV  has  analyzed  and  evaluated  the 
conclusions  and  proposed  implementation  approach  of  each  of  the  contractors.  Based  on 
these  results  and  on  DREV  expertise  in  the  MSDF  domain,  this  report  presents  the  final 
results  on  the  feasibility  and  usefulness  of  an  MSDF  implementation  for  the  CP- 140. 
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Defence  Research  Establishments,  both  in  Ottawa  (DREO)  and  Atlantic  (DREA),  have 
been  consulted  as  relevant  sources  of  specialized  expertise  in  sensors  (radar,  ESM, 
acoustics). 

This  study  addresses  the  analysis  and  evaluation  of  data  fusion  algorithms 
appropriate  for  target  kinematics  (i.e.  position,  speed,  etc.)  and  non-kinematics  (i.e.  shape, 
type,  identity,  etc.)  data  obtained  from  the  CP- 140  multiple  and  dissimilar  sensor  sources 
(e.g.,  radar.  Electronic  Support  Measures  (ESM),  Magnetic  Anomaly  Detector  (MAD), 
sonobuoys,  Forward  Looking  Infra-Red  (FLIR),  Synthetic  Aperture  Radar  (SAR),  etc.), 
and  other  information  available  on-board  the  CP- 140  aircraft,  within  its  operational  and 
tactical  environments,  and  environmental  conditions.  It  is  very  important  to  stress  that  the 
scope  of  this  study  is  limited  to  feasibility  rather  than  detailed  design. 

Relevant  sensor  fusion  concepts  and  terminology  are  defined  in  Chapter  2  of  this 
report,  followed  in  Chapter  3  by  a  definition  of  the  CP- 140  operational  environment. 
Chapter  4  describes  the  CP- 140  information  sources.  A  description  of  applicable  sensor 
fusion  processes  appears  in  Chapter  5.  Expected  performance  improvements  resulting 
from  data  fusion  are  discussed  in  Chapter  6,  and  an  implementation  plan  is  proposed  in 
Chapter  7. 

The  research  and  development  activities  described  in  this  document  were  performed 
at  DREY  between  1994  and  1995  under  PSC32D58. 
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2.0  SENSOR  DATA  FUSION  CONCEPTS  AND  TERMINOLOGY 

Throughout  the  1980s,  the  three  U.S.  military  services  pursued  the  development  of 
tactical  and  strategic  surveillance  systems  employing  data  fusion  and  supported  extensive 
research  in  the  areas  of  target  tracking,  target  identification,  algorithm  development  for 
correlation  (association)  and  classification,  and  the  application  of  intelligent  systems  to 
situation  assessment  (Ref.  1).  The  large  amount  of  fusion-related  work  in  this  period 
raised  some  concern  over  possible  duplication  of  effort.  As  a  result,  the  Joint  Directors  of 
U.S.  Department  of  Defense  (DoD)  Laboratories  (JDL)  convened  a  Data  Fusion  Subpanel 
to  (1)  survey  the  activities  across  all  services,  (2)  establish  a  forum  for  the  exchange  of 
research  and  technology,  and  (3)  develop  models,  terminology  and  a  taxonomy  of  the 
areas  of  research,  development  and  operational  systems. 

As  a  result  of  many  years  of  effort  to  establish  standardization  and  stability  in  the 
lexicon  of  data  fusion,  the  definition  of  many  terms  is  slowly  achieving  consensus  across 
the  diversified  application  community  (Ref.  2).  Problem-specific  nuances  and  shading  in 
these  definitions  remain  but  agreement  on  a  meaningful  subset  of  terms  does  seem  to  exist, 
providing  an  important  basis  for  communication  across  specialized  research  groups. 

This  chapter  provides  a  definition  of  the  basic  terminology  and  concepts  that  are 
used  throughout  the  rest  of  this  report.  Figure  1  while  presenting  the  three  types  of  MSDF 
architecture  that  are  discussed  in  section  2.7,  is  used  to  illustrate  the  MSDF  terminology 
and  concepts. 

2.1  Overall  MSDF  System 

The  overall  tracking  system  (or  MSDF  system)  comprises  both  a  measurement  (or 
sensing)  component  which  provides  observations  of  the  target  environment,  and  an 
estimation  (or  tracking)  component  (ranging  from  a  simple  software  tracking  filter  to  a 
sophisticated  multi-target  multi-sensor  data  fusion  system)  which: 
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1)  acquires  and  maintains  unambiguous,  stable  tracks  corresponding  to  the 
perceived  population  of  real  objects  within  the  volume  of  interest, 

2)  estimates  the  state  and  identity  of  each  tracked  object  with  the 
objective  of  keeping  an  accurate  and  complete  awareness  of  the 
external  environment,  and 

3)  suppresses  clutter  and  other  unwanted  objects  (i.e.,  discards 
"uninteresting"  targets  from  the  scene). 

The  quality  of  the  estimated  picture  (i.e.,  the  output  of  the  data  processing  function) 
should  in  principle  be  superior  to  that  of  the  measured  picture  (i.e.,  the  output  of  the  signal 
processing  function).  As  discussed  in  section  2.7,  there  is  no  particular  requirement  that 
the  measurement  and  tracking  components  be  collocated. 

The  remainder  of  this  chapter  discusses  in  details  the  various  components  of  the 
overall  MSDF  system. 

2.2  Environment 

The  ground  truth  picture  represents  the  real  composition  and  status  of  a  scenario  of 
tactical  interest  in  the  environment.  It  depicts  the  activities  (i.e.,  position,  kinematic 
behavior,  emissions,  and  identity)  of  a  number  of  real,  distinct  targets  in  a  given  area  of 
interest.  A  target  may  be  a  plane,  ship,  missile,  etc.  Targets  possess  defined  kinematic  and 
non-kinematic  properties.  The  trajectory  of  a  target  typically  summarizes  its  kinematic 
properties.  The  target  type  or  category,  its  allegiance,  nationality,  threat  level  and  specific 
identification  are  examples  of  non-kinematic  properties.  The  ground  truth  targets  progress 
in  time  and  space,  defining  a  scenario  that  also  includes  the  characteristics  of  the  target 
emissions. 

The  MSDF  system  has  to  operate  in  the  four-dimensional  world  of  space  and  time. 
Typically,  the  commander  of  a  military  platform  has  to  assess  all  the  ongoing  activities 
within  a  given  space  volume  surrounding  the  platform.  This  volume  is  referred  to  as  the 


Feedback  from 
Sensor  Management 
Function 

•  Cueing 

•  Hand-Off 

•  Sensor  Integration 


/  Environment  \  4 
f  Targets  \ 

•  Kinematics  Properties: 

-  Position 

-  Velocity 

-  Acceleration 
-Manoeuvers 

•  Non-Kinematics  Properties: 

-Type 

-Class 

-  Identity  j 

\-Etc.  / 

\  Scenario  /  \ 


Energy 

Processing 

Video 

Signal 

Signal  Processing 

•  Automatic  Target  Detection  (ATD) 

-  Thresholding  (Detections) 

(Meas.) 

Contacts 

Data 

Processing 

(Estimates) 

Tracks 

•Ete.  Q 

CD 

_ ^  O 

•  Transmitter 

•  Receiver 

•  Antenna  Position 
•Mode 

— r** 

i 

i 

> 

-  M-out-of-N  Detector  (Declarations) 

-  Centroider  (Contacts) 

•  Features  Extraction 

-  Image  Processing  (Classification) 

1  * 

1 

1 

i 

•  Single  Sensor  Tracker 

•  Automatic  Target 

-  Tracking 

-  Identification 

. I . . 

1 

1 

i 

Sensor  1 


Signal-Level 

(Pixel-Level) 


Contact-Level 

(Central-Level) 


Track-Level 

(Sensor-Level) 


Sensor  2 


Energy 

1 

Signal  Processing 

1 

Data 

Processing 

•Transmitter 
•  Receiver 

1 

■  1  ». 
Video 

•  Automatic  Target  Detection  (ATD) 

-  Thresholding  (Detections) 

-  M-out-of-N  Detector  (Declarations) 

-  Centroider  (Contacts) 

1 

1 . 

Contacts 

Processing 

•  Single  Sensor  Tracker 

•  Automatic  Target 

•  Antenna  Position 
•Mode 

Signal 

*  Features  Extraction 
-  Image  Processing 

(Meas.) 

-Tracking 
-  Identification 

Tracks 

(Estimates) 


Fusion 

Function 

MS  DF  Tracker 

•  Alignment 

•  Correlation 

•  Data  Fusion 

•  State  Estimation 

•  Kinematic  Behavior 

•  Features  Extraction 

•  Identity  Fusion 

•  Track  Management 

•  Ouster  Management 


s  Fused 
Sensor  Data 

(Kinematic  +  ID). 


Feedback  from 
Sensor  Management 
Function 


FIGURE  1  -  Definition  of  three  types  of  MSDF  architecture  (i.e.,  signal,  contact  and  track-level)  for  two  generic  sensors 
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Volume  Of  Interest  (VOI).  For  an  MSDF  system,  it  is  defined  by  mission  considerations, 
and  there  is  no  guarantee  that  the  VOI  will  correspond  to  the  MSDF  system's  sensor 
coverages.  Since  one  is  almost  certainly  concerned  with  a  dynamic  environment,  one 
needs  to  take  time  into  account  as  well. 

Any  realistic  physical  tracking  environment  also  includes  a  number  of  unwanted 
objects  of  no  immediate  interest,  e.g.,  sea  surface,  ground,  mountains,  birds,  insects, 
clouds,  rain  and  other  meteorological  phenomena,  etc.  These  objects  may  cause  returns 
that  "clutter"  the  sensor  display  with  false  target  declarations  that  may  overload  the 
processor  elements  and/or  desensitize  the  sensor  to  the  true  targets.  The  important 
performance  degradation  effects  potentially  introduced  by  these  unwanted  objects  must  be 
taken  into  account  by  the  sensor  and  MSDF  designers. 

2,3  Energy  and  Signal  Processing  (Detection  and  Measurement) 

The  measurement  process  encompasses  both  the  energy  and  signal  processing 
aspects.  As  part  of  signal  processing,  target  detection  is  the  process  of  determining  the 
presence  of  a  target,  usually  by  declaring  a  target  present  if  a  voltage  exceeds  a  threshold. 
Typically,  military  sensors  generate  detections  that  involve  "point"  targets  (i.e.,  threshold 
crossings  in  only  a  few  resolution  cells),  from  which  certain  targets  properties  are 
measured.  This  is  discussed  in  more  depth  below. 

A  sensor  will  typically  spend  a  limited  amount  of  time  on  a  single  target  because,  in 
most  cases,  scanning  is  necessary  in  order  to  provide  updated  information  on  established 
tracks  and  to  search  for  new  targets.  One  important  sensor  design  consideration  is  the 
selection  of  a  decision  rule  on  the  return  received  during  the  time  on  target,  so  as  to 
discriminate  between  returns  from  targets  of  interest  and  returns  from  extraneous  sources 
such  as  clutter.  A  widely  spread  approach  to  this  decision  process  is  to  compare  the 
incoming  signal  power  to  a  threshold  which  is  typically  set  so  that  the  probability  of  false 
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alarm  remains  constant.  A  "detection"  occurs  each  time  the  received  power  exceeds  the 
selected  threshold. 

For  a  given  threshold  setting,  the  probability  of  target  detection  will  generally  be  a 
complex  function  of  the  sensor  capabilities,  the  target  size,  the  sensor-target  geometry, 
and  the  physical  environment  (atmospheric  attenuation,  etc.).  The  threshold  value  should 
be  selected  taking  into  account  its  effect  on  overall  tracking  system  performance.  It  may 
even  be  desirable  to  set  the  threshold  adaptively. 

Typically,  it  is  assumed  that  the  measurement  set  produced  by  a  given  sensor  during 
a  single  scan  contains  at  most  one  observation  from  each  target  which  may  be  within  the 
search  volume  of  this  sensor.  This  may  require  some  redundancy  elimination  logic  in  the 
measurement  preprocessing  so  that  multiple  simultaneous  detections  from  the  same  source 
are  combined.  Typically,  a  sensor  search  volume  may  be  covered  using  two  or  more  bars 
in  which  the  sensor  scans  in  azimuth  angle  while  maintaining  a  fixed  elevation  angle  for 
each  bar.  In  such  a  case,  a  redundancy  elimination  logic  is  required  to  ensure  that 
detections  received  from  the  same  target  on  multiple  bars  are  not  interpreted  as  being  the 
result  of  multiple  targets.  The  result  of  the  centroiding  of  simultaneous  detections  to  form 
a  single  report  of  a  target  is  often  called  a  "contact"  (Fig.  1). 

In  addition  to  combining  multiple  detections  from  a  single  target,  it  is  also  desirable 
to  recognize  when  a  single  observation  was  produced  by  multiple  targets.  For  example, 
radar  measurement  techniques  might  not  be  able  to  resolve  several  closely  spaced  targets 
that  are  within  the  radar’s  beamwidth.  However,  radar  data  processing  techniques  have 
been  developed  to  determine  when  there  are  multiple  targets  within  the  radar's  beamwidth, 
even  if  distinct  measurements  from  all  targets  cannot  be  obtained. 

The  term  measurement  usually  refers  to  a  physical  observation  of  a  parameter  (i.e.,  a 
parameter  plus  noise).  In  the  sensor  data  fusion  domain,  measurements  are  thus  noise- 
corrupted  observations  related  in  a  specified  way  to  the  state  of  a  target.  Measurement  is  a 
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collective  term  that  is  used  to  refer  to  all  the  observed  (or  measured)  quantities  included  in 
a  raw  report  (or  a  contact)  output  from  a  sensor.  A  measurement  differs  from  an  estimate 
(or  a  track)  because  an  estimate  operates  on  multiple  measurements  over  time  to  extract  a 
more  accurate  assessment  of  the  parameter. 

Sensor  measurement  characteristics  must  be  defined.  This  involves  specifying 
measurable  parameters,  the  accuracy  associated  with  each  measured  parameter,  and  an 
update  interval  (or  an  adaptive  update  policy)  for  each  sensor.  In  general,  an  observation 
may  contain  measured  kinematic  properties,  such  as  position  or  Doppler  (range  rate),  and 
measured  non-kinematic  properties  (or  attributes)  such  as  target  emitter  type,  radar  cross 
section,  allegiance,  etc.  An  observation  should  also  contain  an  estimate  of  the  time  at 
which  the  measurement  was  obtained  (i.e.,  a  valid  time  tag).  Measurement  accuracies  are 
typically  specified  as  error  variances  or  covariances.  In  general,  observations  may  be 
received  at  regular  intervals  of  time  (scans  or  data  frames),  or  they  may  occur  irregularly 
in  time. 

A  "clean"  measurement  is  of  the  highest  quality;  it  corresponds  to  a  single  object  of 
interest  in  the  environment.  Typically  however,  as  a  result  of  the  many  perturbing  factors 
discussed  in  section  2.9,  the  measured  tactical  picture  is  not  composed  of  solely  clean 
measurements.  A  sensor  can  observe  a  region  containing  an  object  but  fail  to  detect  it  (i.e., 
object  detection  is  not  guaranteed  (probability  of  detection  <  1)),  whether  or  not  it  is  being 
tracked.  Such  undetected  targets  correspond  to  "missed"  measurements.  Sensor 
measurement  techniques  might  not  be  able  to  determine  when  there  are  several  closely 
spaced  targets  within  the  sensor's  beamwidth.  In  such  a  situation,  distinct  measurements 
from  all  targets  cannot  be  obtained,  and  a  single  observation  is  typically  produced  by  the 
multiple  targets.  The  measurement  produced  by  unresolved  closely  spaced  objects  is  often 
called  a  "merged"  measurement  or  a  "clump".  Finally,  some  "spurious"  measurements  (or 
spurious  threshold  crossings)  are  due  to  noise  alone  (i.e.,  false  alarms),  or  due  to  non¬ 
zero-mean  interference  with  unknown  spatial  and  temporal  covariance  (i.e.,  clutter).  These 
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spurious  contacts  can  be  misclassified  later  by  the  estimation  process  as  being  from  an 
object  of  interest. ' 

2.4  Sensor-Level  Data  Processing  (Tracking  and  Identification) 

The  perception  of  the  ground  truth  tactical  picture  by  the  sensor-level  data 
processing  system  is  embodied  in  the  estimates  (or  tracks)  that  are  established  or 
continued  as  the  sensor  samples  the  environment  over  some  time  interval  of  interest.  For 
each  target  in  the  environment,  the  system  attempts  to  maintain  estimates  of  its  location, 
velocity  and  acceleration,  and  several  types  of  attribute  and  identity  estimates. 

Estimation  is  the  process  of  inferring,  in  some  optimal  fashion,  the  value  of  a 
parameter  of  interest  from  indirect  and  inaccurate  measurements  (reported  as  contacts) 
related  in  a  specified  way  to  this  parameter.  In  other  words,  an  optimal  estimator  is  a 
computational  algorithm  that  processes  noisy  measurements  to  obtain  a  "best  estimate" 
(minimum  error  in  some  sense)  of  a  given  parameter  (or  set  of  parameters)  of  interest. 
This  parameter  can  be: 

•  a  time-invariant  quantity  (a  scalar  or  a  vector), 

•  the  state  of  a  dynamic  system  (usually  a  vector ). 

In  principle,  the  estimate  should  be  a  more  accurate  assessment  of  the  parameter 
than  the  raw  measurements.  However,  this  is  not  always  the  case.  The  achievement  of  an 
estimate  of  the  state  of  a  dynamic  system  utilizes  a  priori  information  (or  static  inputs) 
such  as: 


•  knowledge  of  system  and  measurement  dynamics, 

•  assumed  statistics  of  system  noises  and  measurement  errors, 

•  initial  conditions. 


so  that  optimal  estimators  are  sensitive  to  erroneous  a  priori  models  and  statistics. 
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Target  state  estimation  (the  heart  of  any  tracking  system)  is  the  processing  of  noisy 
contact  data,  hypothesized  as  arising  from  the  same  object  or  target,  in  order  to  maintain 
an  estimate  of  its  current  state  which  typically  consists  of  kinematic  components.  State 
estimation  may  consist  of  filtering  (estimating  the  properties  at  the  time  of  the  latest 
observation),  smoothing  (estimating  the  properties  at  a  time  in  the  past  using  all  the 
measurements  available  up  to  the  latest  observation),  and  prediction  (estimating  the 
properties  at  a  time  in  the  future). 

The  target  identification  aspect  also  needs  to  be  considered  in  order  to  produce  the 
complete  tactical  picture.  The  estimation  process  must  accurately  integrate  the 
distinguishing  attributes  of  the  targets  actually  observed,  and  provide  estimates  of  their 
identification. 

Strictly  speaking,  a  track  is  thus  assumed  to  be  a  triple  comprising: 

1)  One  or  more  state  vectors  estimating  the  kinematic  properties  (i.e., 
position,  course  and  speed,  acceleration,  etc.)  of  the  target,  with  a 
covariance  matrix  for  each  state  vector.  If  more  than  one  state  vector  is 
used,  the  relative  likelihood  (or  weight)  of  each  one  is  also  included. 

2)  One  or  more  propositions  about  non-kinematic  properties  (target  type, 
class,  identity,  radar  cross-section,  IR  signature,  etc.)  of  the  target, 
each  with  its  associated  likelihood  function. 

3)  The  probability  of  the  track.  It  is  the  data  processing  algorithm's 
estimate  of  the  absolute  likelihood  that  the  track  exists  (i.e.,  actually 
corresponds  to  some  ground  truth  target). 

In  practice  however,  the  information  kept  in  a  track  file  also  includes  many  other 
parameters  such  as  the  ones  listed  in  TABLE  I  below. 

A  "clean",  stable  track  is  of  the  highest  quality;  it  corresponds  to  a  single  object  of 
interest  in  the  environment  over  the  track's  entire  history,  without  any  pathologies  such  as 
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misassociation  (except,  perhaps,  during  its  initialization  stage,  when  some  bit  of  "bouncing 
around"  may  occur)  or  premature  loss  of  track.  Spurious  tracks  include  "redundant"  (i.e., 
more  than  one  track  for  one  target),  "false"  (i.e.,  tracks  for  no  targets  whatsoever),  and 
"lost"  tracks.  "Missed"  tracks  are  targets  without  tracks. 

In  the  typical  scenarios  we  are  interested  in,  there  can  be  anywhere  from  a  few  to 
hundreds  of  targets  to  follow.  The  tracking  system  must  accurately  indicate  the  correct 
number  of  targets  present.  It  is  important  that  the  tracker  strikes  a  balance  between  too 
many  (false)  and  too  few  (missed  faint  but  real  targets)  tracks. 


TABLE  I 

Typical  track  information  maintained  in  a  track  database 


Parameter  Type 

Example 

Indexing  Parameters  (Book  Keeping) 

track  number,  cluster  number,  track  status 
(i.e.,  potential,  tentative  or  firm),  etc. 

Timing  Information 

time  of  the  current  estimate,  last  update  time, 
etc. 

Quality  Parameters 

quality  index,  score,  etc. 

Update  History  Information 

last  updates  (contacts  or  tracks)  vector, 
hit/miss  pattern  (blip/scan  information),  etc. 

Matching  History  Information 

similarity  with  other  tracks,  etc. 

Data  Association  Information 

validation  matrix,  type  of  gating,  etc. 

Kinematic  Information 

state  vector  and  its  covariance  matrix 

Identity  Information  (Attributes) 

target  size,  shape,  degree  of  symmetry,  etc. 
and  their  confidence  values 

Identity  Information  (ID  Declarations) 

threat  category,  classification,  category, 
description,  type,  class,  unit,  etc.  and  their 
confidence  values 

2.5  Data  Fusion  Definition 

Data  fusion  is  fundamentally  a  process  designed  to  manage  (i.e.,  organize,  combine 
and  interpret)  data  and  information,  obtained  from  a  variety  of  sources,  that  may  be 
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required  at  any  time  by  operators  and  commanders  for  decision  support.  The  sources  of 
information  may  be  quite  diverse,  including  sensor  observations,  data  regarding  capability 
and  availability  of  targets,  topographic  and  environmental  data,  and  information  regarding 
doctrine  and  policy.  The  data  and  information  provided  by  these  various  sources  may 
contain  numbers  of  targets,  conflicting  reports,  cluttered  backgrounds,  degrees  of  error, 
deception,  and  ambiguities  about  events  or  behaviors. 

In  this  context,  Data  Fusion  (DF)  is  an  adaptive  information  process  that 
continuously  transforms  the  available  data  and  information  into  richer  information, 
through  continuous  refinement  of  hypotheses  or  inferences  about  real-world  events,  to 
achieve  refined  (and  potentially  optimal)  kinematic  and  identity  estimates  of  individual 
objects,  and  complete  and  timely  assessments  of  current  and  potential  future  situations  and 
threats  (i.e.,  contextual  reasoning),  and  their  significance  in  the  context  of  operational 
settings. 

The  process  is  also  characterized  by  continuous  refinements  of  its  estimates  and 
assessments,  and  by  evaluation  of  the  need  for  additional  data  and  information  sources,  or 
modification  of  the  process  itself,  to  achieve  improved  results. 

2.6  Data  Fusion  Hierarchy 

The  process  of  data  fusion  may  be  viewed  as  a  multi-level  hierarchical  inference 
process  whose  ultimate  goal  is  to  assess  a  mission  situation  and  identify,  localize  and 
analyze  threats.  However,  not  every  data  fusion  application  is  responsible  for  all  of  these 
outputs.  Some  applications  are  only  concerned  with  the  position  and  identification  of 
objects.  Other  applications  are  primarily  oriented  towards  the  situation  and  how  it  is 
evolving.  Still  others  focus  on  the  threat  and  its  possible  impacts  upon  achieving  mission 
objectives.  In  addition,  the  data  fusion  function  can  be  responsible  for  identifying  what 
information  is  most  needed  to  enhance  its  products  and  what  sources  are  most  likely  to 
deliver  this  needed  information. 

Given  these  considerations,  a  complete  data  fusion  system  can  typically  be 
decomposed  into  four  levels: 

Level  1  -  Multi-Sensor  Data  Fusion  (MSDF); 

Level  2  -  Situation  Assessment  (SA); 
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Level  3  -  Threat  Assessment  (TA);  and. 

Level  4  -  Process  Refinement  Through  Resource  Management  (RM). 

Each  succeeding  level  of  data  fusion  processing  deals  with  a  greater  level  of 
abstraction.  Level- 1  data  fusion  uses  mostly  numerical,  statistical  analysis  methods,  while 
level-2, 3  and  4  data  fusion  use  mostly  symbolic,  Artificial  Intelligence  (AI)  methods.  Note 
that  resource  management  in  the  context  of  level-4  fusion  is  mainly  concerned  with  the 
information  gathering  process  refinement  (i.e.,  sensor  management).  The  overall  domain 
of  resource  management  also  encompass  the  management  of  weapon  systems. 

2.6.1  Level  1  -  Multi-Sensor  Data  Fusion 

Multi-sensor  data  fusion  (MSDF)  is  concerned  solely  with  individual  objects,  first  in 
associating  the  sensor  outputs  with  specific  known  objects  or  using  them  to  initiate  new 
objects.  Level- 1  processing  uses  sensor  data  to  correctly  and  quickly  derive  the  best 
estimates  of  current  and  future  positions  for  each  hypothesized  object.  In  addition, 
inferences  as  to  the  identity  of  the  objects  and  key  attributes  of  the  objects  are  developed. 

Key  MSDF  functions  include:  data  alignment,  data  association  or  correlation, 
kinematic  data  fusion,  target  state  estimation,  target  kinematics  behaviour  assessment, 
target  identity  information  fusion  and  the  management  of  clusters  and  tracks. 

In  any  MSDF  system,  sensor  data  alignment  in  time  and  space  must  take  place 
before  state  estimation  can  be  performed.  Moreover,  in  order  to  estimate  and  remove  the 
effects  of  sensor  motion  from  the  received  data,  various  Inertial  Navigation  Systems  (INS) 
are  used,  involving  a  wide  variety  of  motion  sensors  including  gyroscopes,  accelerometers, 
and  the  Global  Positioning  System  (GPS)  (Ref.  3).  The  motion  corrected  observations  are 
processed  to  form  tracks. 

The  functions  of  data  association  (labeling  measurements  from  different  origins 
and/or  sensors,  at  different  times,  that  correspond  to  the  same  object  or  feature)  and  data 
fusion  (combining  measurements  from  different  times  and/or  different  sensors)  are  also 
required  in  one  form  or  another  in  essentially  all  multiple  sensor  fusion  applications:  one 
function  determines  what  information  should  be  fused,  the  other  function  performs  the 
fusion  (Ref.  3). 
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2.6.2  Level  2  -  Situation  Assessment 

Based  on  incomplete  and  inaccurate  sets  of  data  and  information,  situation 
assessment  (SA)  is  devoted  to  the  continuous  inference  of  statements  about  the 
hypothesized  objects  provided  by  the  lower  level  data  fusion  function  in  order  to  derive  a 
coherent,  composite  tactical  picture  of  the  situation.  This  picture  must  be  described  in 
terms  of  groups  or  organizations  of  objects  so  that  enemy  intent  can  be  estimated  in  the 
next  level  and  decisions  can  be  made  by  decision  makers  about  how  to  use  war  fighting 
assets. 

Hence,  SA  fits  hypothesized  objects  with  known  and  expected  organizations  and 
events,  together  within  the  constraints  of  terrain  and  enemy  tactics,  to  develop  a 
description  or  interpretation  of  the  current  relationships  among  these  objects  and  events  in 
the  context  of  the  operational  environment.  The  result  of  this  processing  is  a  determination 
or  refinement  of  the  battle/operational  situations.  Based  on  the  situation  abstraction 
products  and  information  from  technical  and  doctrinal  databases,  SA  also  attempts  to 
anticipate  future  events  over  a  short  time  horizon.  Key  SA  functions  include:  object 
aggregation,  event/activity  aggregation,  contextual  interpretation/fusion  and  multi¬ 
perspective  assessment. 

2.6.3  Level  3  -  Threat  Assessment 

Threat  assessment  (TA)  is  focused  at  the  details  necessary  for  decision  makers  to 
reach  conclusions  about  how  to  position  and  commit  the  friendly  forces. 

By  coupling  the  products  of  situation  assessment  with  the  information  provided  by  a 
variety  of  technical  and  doctrinal  databases,  TA  develops  and  interprets  a  threat  oriented 
perspective  of  the  data  to  estimate  the  enemy  capabilities  and  lethality,  identify  threat 
opportunities  in  terms  of  the  ability  of  own  force  to  engage  the  enemy  effectively,  estimate 
enemy  intent  (i.e.,  provide  indications  and  warnings  of  enemy  intentions),  and  determine 
levels  of  risk  and  danger. 

Hence,  TA  uses  the  situation  picture  from  level  2  and  what  is  known  about  the 
enemy  doctrine  and  objectives  to  predict  the  strengths  and  vulnerabilities  for  the  threat 
forces  and  friendly  forces.  In  addition,  the  friendly  mission  and  specific  options  available  to 
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the  decision  makers  are  tested  within  these  strengths  and  vulnerabilities  to  guide  decision 
making. 

Key  TA  functions  include:  enemy  forces  capability  estimation,  predict  enemy  intent, 
identify  threat  opportunities,  multi-perspective  assessment  and  offensive/defensive 
analysis. 

2.6.4  Level  4  -  Process  Refinement  (Sensor  Management) 

Information  resource  management,  level  4  processing,  closes  the  loop  by  first 
examining  and  prioritizing  what  is  unknown  in  the  context  of  the  situation  and  threat  and 
then  developing  options  for  collecting  this  information  by  cueing  the  appropriate  sensors 
and  collection  sources. 

2.7  Multi-Sensor  Data  Fusion  Architectures 

Before  an  MSDF  function  can  be  implemented  within  a  military  platform,  it  must  be 
analyzed  in  terms  of  the  different  types  of  architecture  and  implementations  that  are 
possible,  the  benefits  and  drawbacks  of  these  implementations,  and  finally  in  terms  of  how 
all  this  relates  to  the  performance  and  mission  requirements  of  the  platform  (Ref.  4). 

For  any  given  sensor  suite  configuration,  there  can  be  many  different  ways  to 
combine  data  from  the  sensors.  The  term  "MSDF  architecture"  is  used  to  indicate,  based 
on  the  level  at  which  the  sensor  data  are  fused  (i.e.,  signal,  contact  or  track  level),  the 
general  method  (or  philosophy)  used  to  combine  the  sensor  data  into  global  tracks  within 
an  MSDF  function. 

Figure  1  illustrates  on  a  single  diagram  the  usual  definition  of  three  types  of  MSDF 
architecture  for  two  generic  sensors.  One  possible  type  of  MSDF  architecture  is  based  on 
maintaining  sensor-level  tracks  at  each  sensor  site,  finding  the  sensor  tracks  that 
potentially  represent  the  same  target  and  then  combining  these  tracks  into  global  tracks  of 
the  MSDF  function.  A  second  type  of  architecture  assumes  that  the  raw  sensor 
measurements  (i.e.,  sensor  contacts)  are  sent  directly  to  the  MSDF  function  to  be 
combined  into  global  tracks.  This  architecture  is  sometimes  referred  to  as  a  "central-level" 
architecture  since  the  tracks  are  only  formed  into  the  central  processor.  As  a  third  type, 
fusion  at  the  signal-level  typically  combines  signals  from  similar  sensors  to  produce  a 
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better  quality  signal  of  the  same  form.  Finally,  there  is  also  the  possibility  of  using  a 
mixture  of  these  three  types  of  architecture  to  form  a  hybrid  (or  combined)  tracking 
structure. 

The  MSDF  architecture  is  an  important  issue  since  the  benefits  are  different 
depending  on  the  way  the  sensor  data  are  combined.  The  selection  of  the  MSDF 
architecture  type  should  be  aimed  at  optimizing  the  target  detection,  tracking  and 
identification  performance  required  for  a  specific  platform  given  its  missions.  However, 
the  selection  is  also  constrained  by  the  technological  capabilities  (both  hardware  and 
software).  It  depends  on  the  quality  of  the  sensors  being  fused,  the  availability  of  computer 
processing  power,  the  bandwidth  of  the  available  data  transmission  paths,  and  the  degree 
to  which  operator  intervention  is  required  or  desired  (see  Refs.  5-6).  Table  II  below, 
extracted  from  Ref.  7,  lists  the  main  advantages  of  the  central-level  architecture. 

TABLE  n 

Advantages  of  the  central-level  fusion  architecture 


Advantage 

Reason 

Quick  Reaction  Time 

All  sensors  which  detect  a  target  will  be  used  to  initiate 
track  on  the  target. 

Optimum  Track 

Accuracy 

Each  track  contains  more  data  per  unit  time.  Data  from 
one  sensor  may  be  used  to  augment  data  from  another 
sensor. 

Good  Track  Continuity 

Each  track  contains  more  data  per  unit  time.  Also  one 
sensor  may  be  blocked  while  another  has  contact  with 
the  target  being  tracked. 

Good  Track  Resolution 

Sensors  have  different  resolving  abilities.  The  angle 
resolution  of  a  FLIR  is  10  times  as  good  as  a  radar. 

Good  False  Track 
Suppression 

The  data  from  each  sensor  is  used  immediately  to 
confirm  target  maneuvers  before  track  is  lost  or 
corrupted. 

The  disadvantages  of  central-level  fusion  are  that  it  requires  a  large  amount  of 
computer  processing  power,  a  relatively  high  bandwidth  for  transmitting  contact  data  from 
the  sensor  to  the  central  site  for  fusion,  and  it  must  be  possible  to  correlate  the  sensor  data 
to  the  track  data  on  a  scan-to-scan  basis. 
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The  advantages  and  disadvantages  of  sensor-level  fusion  are  the  mirror  image  of 
the  contact-level  fusion  discussed  above.  This  approach  requires  less  processing  power  in 
the  central  data  fusion  site  and  requires  less  I/O  bandwidth  to  transmit  the  data  between 
the  sensor  and  the  central  site.  Table  III  below,  also  extracted  from  Ref.  7,  lists  the  main 
disadvantages  of  the  central-level  architecture. 


TABLE  m 

Disadvantages  of  the  sensor-level  fusion  architecture 


Disadvantage 

Reason 

Poor  Reaction  Time 

A  track  is  detected  only  after  a  single  sensor  has  promoted 
the  track  to  firm 

Poor  Track  Accuracy 

If  track  selection  fusion  is  used  then  single  sensor  tracks 
will  not  have  the  accuracy  of  multiple  sensor  tracks.  If 
track  fusion  is  used  then  target  maneuvers  become  difficult 
to  follow. 

Poor  Track  Continuity 

Track  selection  fusion  can  lead  to  jittery  (unsmooth)  track 
data  while  track  fusion  may  cause  a  break  in  track  during 
target  maneuvers. 

Poor  Track  Resolution 

It  is  very  likely  that  a  track  which  is  resolved  on  one  sensor 
and  not  on  another  will  appear  as  two  or  three  tracks  in  this 
type  of  fusion. 

Poor  False  Track 
Suppression 

Poor  track  to  track  correlation  will  inevitably  lead  to 
multiple  tracks  on  the  same  target. 

2.8  Sensor  Management 

In  the  traditional  elaboration,  MSDF  is  portrayed  as  a  purely  passive,  open  loop 
process  (i.e.,  a  function  that  simply  processes  whatever  it  receives).  In  the  more  advanced 
and  enlarged  sense  however,  an  MSDF  system  also  includes  many  additional  functions,  the 
most  essential  of  which  is  active  feedback.  An  MSDF  system  not  only  detects,  localizes, 
and  identifies  targets,  but  also,  on  the  basis  of  an  evolving  picture,  manages  the 
information  it  might  receive  by  pointing,  focusing,  manoeuvering,  and  adaptively  selecting 
the  modalities  of  its  sensors  and  sensor  platforms  (Ref.  8). 
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In  order  to  maximize  target  information  and  fire  support  obtained  by  sensors  while 
minimizing  the  threat  to  system  and  assets  (e.g.,  through  an  emission  control  (EMCON) 
policy),  sensor  management  is  a  decision  process  which  addresses  the  following  questions: 

•  When  to  search,  track,  remain  covert? 

•  What  regions  to  search,  what  objects  to  track? 

•  How  long  to  search,  track,  remain  covert? 

•  With  which  sensor  combinations?  Fire  support? 

Sensor  management  is  thus  a  resource  allocation  problem.  Sensor  cueing,  hand-off 
and  scheduling  issues  are  part  of  the  sensor  management  domain. 

The  management  of  sensors  may  require  that  different  sensors  cooperate  to  acquire 
measurements  on  a  common  target.  The  two  primary  cooperative  functions  are  cueing  and 
hand-off.  Cueing  is  the  process  of  using  the  detections  (i.e.,  contact-level  cueing)  or  tracks 
(i.e.,  track-level  cueing)  from  one  sensor  (A)  to  point  another  sensor  (B)  toward  the  same 
target  or  event.  Hand-off  occurs  when  sensor  A  has  cued  sensor  B  for  transferring 
surveillance  or  fire  control  responsibility  from  A  to  B. 

Two  processes  must  occur  for  cueing  or  hand-off:  (1)  the  cueing  sensor  must 
provide  the  cued  sensor  data  that  contains  sufficient  information  to  point  to  the  target  and 
identify  it  as  the  specific  target  being  cued,  (2)  the  cued  sensor  must  search  for  the  target 
of  interest  and  verify  that  it  has  been  acquired. 

Another  issue  related  to  active  feedback  is  sensor  integration  which  involves  the 
modification  of  the  sensor  design  so  that  it  can  receive  and  use  pertinent  information  from 
other  sensors  or  from  the  Command  and  Control  process,  to  improve  or  refine  its  own 
performance.  In  other  words,  sensor  integration  allows  the  sensor  to  do  its  task  better  than 
as  a  stand  alone  autonomous  sensor.  Sensor  integration  is  to  a  large  extent  a  sensor 
system  designer's  issue. 

In  summary,  an  advanced  MSDF  system  indicates  what  the  targets  are,  where  they 
are,  where  they  aren't,  and  where  it  hasn't  looked  (Ref.  8).  In  this  regard,  sensors  and 
sensor  platforms  are  selectively  employed  to: 
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•  look  for,  and  find  targets  within  a  specified  volume  of  interest  (this 
implies  that  one  demarcates  the  VOI,  and  defines  search  strategies  for 
achieving  the  best  possible  sensor  coverage), 

•  enhance  accuracy  (by  point  and  dwell)  against  priority  targets, 

•  increase  detections  (by  more  frequent  visits)  in  interesting  or  threatening 
regions, 

•  balance  these  objectives  in  accordance  with  the  mission  declaration,  and 

•  operate  its  sensors  within  power,  time,  mutual  interference,  and  EMCON 
constraints. 

In  practice,  most  current-day  MSDF  systems  achieve  feedback  through  the  agency 
of  operators  equipped  with  tactical  decision  aids.  These  functions  can  be  called 
"Value/Cost  Analysis,  Decision  and  Command".  With  the  introduction  of  next-generation 
sensors,  characterized  by  receiver-transmitter  agility,  abundant  modalities,  and 
multifarious  constraints,  automation  of  the  sensor  management  and  integration  loop  will 
become  a  virtual  necessity  (Ref.  8). 

2.9  Limitation  Factors 

The  observations  (i.e.,  the  contacts)  generated  by  the  energy  and  signal  processing 
processes  and  passed  to  the  tracker  are  affected  by  the  characteristics  of:  the  targets  of 
interest,  other  objects  in  the  field  of  view,  background  clutter,  environmental  phenomena 
between  sensor  and  objects/background,  sensor  location  relative  to  the  objects  and 
background,  design  of  the  sensors  and  signal  processor  algorithms,  etc.  The  measurements 
may  also  be  distorted  by  sensor  pointing  and  location  (navigation)  errors.  The  resulting 
measurement  vectors  thus  reflect  feature  inferencing  errors,  kinematic  measurement  errors 
that  typically  are  not  Gaussian,  false  and  missing  observations,  impact  of  background 
clutter,  unresolved  closely  spaced  objects,  etc. 

A  number  of  uncertainties  also  affects  the  data  processing  function  (i.e.,  the 
estimation  process).  Target  state  estimation  algorithms  typically  use  some  practical  models 
of  target  motion  in  order  to  estimate  the  present  and  future  target  kinematic  quantities. 
These  target  kinematic  models  are  generally  simple  (such  as  straight  line  paths,  circles, 
etc.)  and  assumed  to  be  described  by  well  known  physical  laws  (e.g.,  ballistic  laws,  etc.). 
Unexpected  changes  to  these  assumed  target  motion  models  (i.e.,  "random"  acceleration) 
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are  called  manoeuvers.  Any  mismatch,  during  a  manoeuver,  between  the  real  kinematic 
behavior  of  a  target  and  the  motion  model  assumed  by  a  simple  target  state  estimation 
algorithm  can  completely  degrade  the  performance  of  the  estimation  technique.  For 
example,  in  addition  to  the  development  of  a  track  bias,  some  measurements  actually  from 
an  object  of  interest  can  be  misclassified  as  being  from  a  different  object,  or  as  being  noise 
or  clutter. 

2.10  Automation  Issues 

The  issues  of  interest  for  the  CP- 140  are  target  detection,  tracking  and  identification. 
Indeed,  accurately  detecting,  locating  and  identifying  potential  targets  is  fundamental  for 
the  success  of  the  mission  of  the  aircraft.  Given  the  diversity  of  the  many  sensors  onboard 
the  CP- 140,  data  fusion  is  mandatory  as  a  means  to  exploit  the  unique  combinations  of 
data  that  is  available.  In  view  of  these  considerations,  a  major  aspect  that  needs  to  be 
addressed  is  the  issue  of  manual  operations  versus  automation.  Figure  2  illustrates  the 
interrelationships  between  these  various  concepts  related  to  sensor  data  fusion. 

Currently,  target  detection,  tracking  and  identification  are  performed  manually  by  the 
operators  onboard  the  aircraft.  Sensor  data  fusion  is  also  performed  manually.  The  scope 
of  the  feasibility  study  discussed  in  this  report  is  the  automation  of  all  these  aspects. 
However,  the  optimal  upgrade  path  from  the  current  system  (where  everything  is  manual) 
to  the  ultimate  automated  system  (including  automated  MSDF)  is  far  from  being  trivial. 
Chapter  7  presents  the  incremental  approach  (with  recoverable  steps)  recommended  by 
DREV  to  implement  sensor  data  fusion  on  the  CP- 140. 

Before  any  of  the  detection,  tracking  and  identification  processes  can  be  automated, 
the  video  signal  from  the  sensor  must  be  digitized.  This  is  a  formality.  It  has  no  impact  on 
operator  efficiency  or  mission  effectiveness. 

Then,  one  may  automate  the  target  detection  process  for  one  or  more  sensors 
individually.  This  could  improve  mission  effectiveness.  However,  in  order  to  be  able  to 
detect  very  small  targets  at  long  range,  the  detection  process  would  have  to  be  allowed  to 
generate  many  false  alarms.  This  could  provide  additional  load  on  the  operator  to 
manually  manage  all  these  detections  (true  and  false),  thus  impacting  the  operator 
efficiency. 
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A  solution  to  this  problem  could  be  to  also  automate  the  tracking  and  identification 
process  at  the  sensor  level  (i.e.,  for  each  sensor  individually).  However,  the  development 
and/or  acquisition  of  such  better  but  stand-alone  sensors  used  in  isolation  from  each  other 
typically  leads  to  a  confusing  and  time-late  decision  environment.  The  operators  might  be 
flooded  by  the  volume,  rate  and  complexity  of  the  information  provided  by  these  sensors 
to  a  point  where  their  ability  to  cope  with  that  information  may  be  exceeded,  thus  also 
impacting  the  operator  efficiency. 

A  key  element  to  this  information  management  problem  is  the  ability  to  automatically 
combine  or  fuse  data  from  the  sensors.  This  is  the  optimal  way  to  do  automatic  target 
detection,  tracking  and  identification  when  multiple  sensors  are  available.  One  might  first 
do  automatic  detection,  tracking  and  identification  with  each  single  sensor  individually  and 
then  do  sensor  fusion.  However,  as  previously  discussed  in  section  2.7,  this  is  not  the 
optimal  way  of  doing  automation  for  a  multiple  sensor  system  since  one  limits  the  fusion 
to  be  at  the  track  level  (as  opposed  to  fusion  at  contact  level  which  is  the  optimal 
approach).  The  optimal  upgrade  path  is  rather  to  automate  the  individual  sensors  up  to  a 
certain  level  (say,  automatic  target  detection),  and  then  to  automate  the  tracking  and 
identification  processes  based  on  a  central-level  MSDF  architecture.  Chapter  7  discusses 
these  issues  in  more  depth,  and  presents  the  recommended  approach. 
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FIGURE  2  -  Interrelationships  between  the  various  concepts  related  to  sensor  data  fusion 
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3.0  DEFINITION  OF  THE  CP-140  OPERATIONAL  ENVIRONMENT 


This  Chapter  summarizes  from  Ref.  9  the  CP- 140  operational  environment  in  terms 
of  mission  requirements  (e.g.,  ASW,  surface  surveillance,  target  tracking,  goals,  etc.)  and 
tactical  scenarios  (targets,  environmental  conditions.etc.).  Loral  Canada  has  identified 
from  Ref.  10  the  missions  to  which  MSDF  techniques  could  be  applied  to  advantage. 
Details  on  the  missions  are  in  Refs.  9-10;  here  we  only  present  the  essential  information 
for  the  sake  of  the  feasibility  study. 

3.1  Mission  Requirements 

The  mission  of  Air  Command  is  to  maintain  balanced,  general  purpose,  combat  capable  air 
forces  to  meet  Canada’s  defence  policy  objectives.  In  supporting  these  objectives,  the 
Aurora  is  called  upon  to  perform  various  mission  elements,  each  of  which  is  supported 
through  the  performance  of  associated  tasks,  which  collectively  account  for  the  major 
General  Purpose  Air  Forces  (GPAF)  activities.  Using  the  nomenclature  of  the  GPAF, 
Loral  Canada  has  identified  the  following  mission  elements  relevant  to  the  CP- 140: 

a.  A3  -  Maritime  Defence 

b.  A6  -  Domestic  Air  Support; 

c.  A7  -  Collective  Defence  of  the  North  Atlantic; 

d.  A8  -  Maintenance  of  International  Peace 

e.  A9  -  Support  of  Canadian  Interests  Abroad 

f.  A12  -  Collective  and  Individual  Training 


In  fulfilling  its  assigned  missions,  in  both  peacetime  and  wartime,  the  Aurora  is  called 
upon  to  perform  those  tasks  defined  in  Table  IV.  A  detailed  description  of  those  tasks  can 
be  found  in  Ref.  9. 
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TABLE  IV 


Aurora  mission  elements  and  associated  tasks  according  to  the  GPAF 


MISSION 

TASKS  H 

A3  -  Maritime  Defence 

AC  3  -  Maritime  Area  Operations 

AC  4  -  Maritime  Direct  Support 

A6  -  Domestic  Air  Support 

AC  1 1  -  Search  and  Rescue 

AC  15  -  Counter  Drug 

AC  16  -  Maritime  Patrols 

AC  17  -  Northern  Patrols 

AC  1 9  -  Domestic  Contingency 

A7  -  Collective  Defence  of  the 
North  Atlantic 

AC  21  -  NATO  Maritime  Operations 

A8  -  Maintenance  of 
International  Peace 

AC  24  -  Air  Continental  Operations 

AC  25  -  Joint  Maritime  Air  Continental  Operations 
AC  28  -  Air  Surveillance 

A9  -  Support  of  Canadian 
Interests  Abroad 

AC  24  -  Air  Continental  Operations 

AC  25  -  Joint  maritime  Air  Continental  Operations 
AC  28  -  Air  Surveillance 

A 12  -  Collective  and 

Individual  Training 

AC  34  -  Operational  Training  Unit  Training 

3.2  Tactical  Scenarios 

Given  the  varied  nature  of  the  present  and  planned  Aurora  tasking,  it  is  impossible  to 
define  a  set  of  mission  scenarios  which  will  describe  all  possible  tasking.  However,  based 
upon  those  defined  in  the  Marconi  Human  Factors  Engineering  Study  (Ref.  10),  the 
following  set  of  scenarios  is  considered  for  the  purpose  of  this  study: 

a.  conduct  nuclear  submarine  asw; 

b.  conduct  diesel-electric  submarine  asw; 

c.  join  task  force; 

d.  conduct  over-the-horizon  targeting  and  damage  assessment; 


P228813.PDF  [Page:  37  of  90] 


UNCLASSIFIED 

25 


e.  conduct  search  and  rescue; 

f.  shadow  a  surface  vessel  suspected  of  smuggling; 

g.  conduct  fishing  and  pollution  surveillance,  and, 

h.  conduct  northern  patrol 


TABLE  V 

List  of  missions/tasks/scenarios  with  expected  targets  and  information 


Mission  /Task 
/Scenario 

Expected  Targets 

Information  Sources 

Fisheries  Patrol 

Medium  sized  boats, 
trawlers 

Radar,  SSAR,  FLIR,  camera 

Drug  Smuggling 

Patrol 

Small  speed  boats,  fishing 
boats,  various  sized  ships 

SSAR,  radar,  FLIR,  camera, 
ESM,  Link-1 1 

Pollution  Surveillance 

Tankers,  large  ships,  small 
boats 

Radar,  SSAR,  FLIR,  camera 

Search  and  Rescue 

Humans,  dinghies,  small 
boats,  various  sized  ships 

Radar,  SSAR,  ESM,  FLIR, 
Link- 11 

Aircrew  Training 

All  types 

All  sensors 

ASuW 

Large  ships 

SSAR,  radar,  FLIR,  IFF, 
acoustics 

Join  Task  Force 

All  types 

All  sensors 

Air  Surveillance 

Air  targets 

IFF,  ESM,  FLIR,  camera 

ASW 

Nuclear  subs,  diesel-electric 
subs 

Acoustics,  ESM,  MAD, 

FLIR,  radar 

Loral  Canada  (Ref.  9)  has  used  its  airborne  personnel’s  expertise  to  estimate  the 
present  frequency  of  occurrence  of  the  various  missions/tasks/scenarios.  The  purpose  was 
not  to  re-prioritize  the  CP- 140  missions  but  only  to  assess  the  benefits  of  data  fusion 
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against  the  most  frequent  missions.  The  missions/tasks/scenarios  are  shown,  in  decreasing 
frequency  of  occurence,  in  Table  V  where  the  targets  that  need  to  be  detected,  tracked 
and  identified  are  listed  by  priority,  for  the  purpose  of  this  study  only,  together  with  the 
main  information  sources.  The  order  becomes  more  arbitrary  as  one  progresses  further 
down  the  list.  This  order  may  change  if  the  Aurora  becomes  more  involved  in  international 
missions.  It  is  apparent  that  when  all  surface  vessels  must  be  considered  as  possible  threats 
and  when  the  CP- 140  aircraft’s  maritime  surveillance  missions  take  higher  precedence,  the 
role  of  sensors  which  can  detect  and  provide  attribute  information  about  surface  objects 
(e.g.  SAR,  FLIR  and  ESM)  has  to  increase. 

3.2.1  Environmental  Conditions 

The  Aurora  operates  in  every  imaginable  extreme  of  weather,  from  arctic  winter 
conditions,  during  northern  patrols,  to  conditions  of  high  temperature  and  high  humidity 
during  deployed  operations  in  the  Caribbean  and  Southern  Pacific  regions.  Since  the 
majority  of  maritime  patrol  activity  occurs  over  the  North  Atlantic  and  Northern  Pacific 
Oceans,  the  weather  normally  encountered  in  these  regions  has  a  great  impact  on  Aurora 
operations.  In  addition,  the  weather  in  the  North  Atlantic  varies  not  only  by  season  but  by 
region. 

Several  special  atmospheric  conditions  can  adversely  affect  the  received  data.  For 
non-acoustic  sensors: 

-  rain  and  snow  attenuate  the  radar  beam  due  to  scattering  in  both  directions  of 
propagation,  while  the  phenomena  have  an  impact  on  only  one  direction  for  the  FLIR 

-  fog  can  mask  surface  targets  from  optical  sensors 

-  atmospheric  refraction  can  yield  propagation  ducts  or  propagation  holes 

-  wind  speeds  in  excess  of  25  knots  reduce  the  radar,  FLIR  and  MAD’s  effectiveness 

-  shallow  waters  provide  a  rough  bottom  structure  and  composition  that  produce  local 
variations  of  the  earth’s  magnetic  field,  thus  confusing  the  MAD 
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-  sunspot  activity  can  affect  propagation  and  MAD 

For  acoustic  sensors  which  are  of  concern  mostly  during  the  ASW  mission,  the  list  is 
equally  imposing: 

-  natural  deep  ocean  noises  of  geological  or  biological  origin  can  provide  a  background 
that  reduces  sonobuoy  effectiveness 

-  rain  or  turbulence  produce  sea  surface  background  noise  and  surface  traffic  further 
confuses  the  ASOs 

-  rough  seas  can  cause  temporary  antenna  immersion  and  therefore  an  intermittent 
Sonobuoy  Radio  Frequency  (RF)  signal 

-  reverberation  effects  (surface,  bottom  or  volume)  cause  unwanted  underwater  signals  for 
active  systems 

-  ocean  currents  can  affect  acoustic  tactics,  e.g.  Gulf  stream  versus  mid-ocean  region  etc.; 
currents  cause  drifts  which  have  to  be  accounted  for  correct  sonobuoy  positioning  and 
create  acoustic  walls 

-  shallow  water  ASW  is  particularly  difficult. 

3.2.2  Electromagnetic  Environment 

During  peacetime,  the  Aurora  generally  operates  overtly,  communicating  over  open 
radio  channels  with  Base  Operations,  Headquarters,  Air  Traffic  Control  and  cooperating 
forces.  Depending  on  the  nature  of  the  mission,  the  Aurora  will  likely  participate  in  tactical 
data  link  with  other  surface  and  air  forces,  providing  over-the-horizon  targeting  and 
receiving  updated  tactical  information.  During  peacetime,  full  use  will  be  made  of  all 
sensors,  both  active  and  passive,  including  extensive  use  of  the  radar  to  locate  surface 
targets. 

During  wartime,  the  ability  of  the  Aurora  to  openly  communicate  with  friendly 
forces  will  be  severely  restricted  by  the  requirement  to  limit  detection  through  the 
enforcement  of  Emission  Control  (EMCON)  conditions.  Communications,  when  essential. 
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will  be  encrypted  and  limited  to  the  briefest  possible  duration.  To  prevent  broadcasting  its 
position,  the  Aurora  will  adopt  a  “receive  only”  posture,  accepting  radio  communications, 
but  not  broadcasting  an  acknowledgment. 

Sonobuoys  will  continue  to  be  the  major  sensor  for  submarine  detection  and  tracking 
during  wartime.  For  the  detection  of  surface  targets,  however,  greater  emphasis  will  be 
placed  on  the  use  of  passive  sensors,  such  as  acoustics,  Forward  Looking  Infrared  (FLIR) 
and  ESM,  with  radar  emissions  restricted  to  a  single  sweep  in  order  to  confirm  target 
location  prior  to  attack. 

Depending  on  the  field  of  operations  and  the  nature  of  the  enemy  forces,  the  Aurora 
may  be  subjected  to  electromagnetic  jamming  in  an  attempt  to  confuse  its  communications 
and/or  its  radar  and  overload  its  passive  sensors.  Under  such  conditions,  the  jamming 
agent  becomes  highly  visible  in  the  electromagnetic  spectrum,  generally  requiring 
extensive  protection  in  order  to  remain  on  the  air.  For  maritime  patrols,  it  is  most  likely 
that  the  Aurora  will  encounter  totally  covert  enemy  forces  as  opposed  to  an  active 
jamming  environment. 
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4.0  CP-140  INFORMATION  SOURCES 

An  analysis  of  the  information  sources  for  the  CP- 140  aircraft  is  presented  in  this 
chapter.  The  analysis  covers  the  organic  sensor  information  from  the  current  sensor  suite 
versus  the  information  from  an  advanced  suite  of  the  same  type  of  sensors,  information 
available  from  additional  sensors  onboard  the  aircraft,  and  sensor  information  from  other 
non-organic  sources. 

The  investigation  also  contains  an  analysis  of  the  current  non-sensor  technology  for 
providing  information  to  be  used.  Non-sensor  technology  is  defined  as  all  knowledge 
sources  other  than  sensors,  that  can  provide  kinematic,  identity  or  other  information  (e.g., 
data  links,  intelligence  reports,  environmental  data,  visual  sightings  (  i.e.,  the  “human 
eyeballs”),  encyclopaedic  data,  etc.). 

4.1  Review  of  the  Current  CP-140  Sensor  Suite 

Each  sensor  on  board  the  CP- 140  has  to  be  described  in  terms  of  its  potential 
contribution  to  the  estimation  of  the  state  vector  (kinematic  and  ID)  of  each  target.  This 
idealized  contribution  will  be  affected  by  the  confronted  tactical  environment  and  the 
differing  environmental  conditions.  A  more  specific  definition  of  the  information  sources 
interface  with  the  MSDF  process  is  given  in  Chapter  5. 

The  following  list  (Table  VI)  is  an  account  of  the  present  sensors  (in  rough  order  of 
importance  to  fusion),  with  their  measured  kinematic  and  non-kinematic  data  (including 
measurements  accuracy  where  applicable).  The  identity  information  that  can  be  obtained 
through  further  processing  is  also  provided  in  the  list.  The  information  is  extracted  from 
both  Refs.  9  and  11.  Here  we  retain  only  the  essential  information  relevant  to  data  fusion. 

In  Table  VI  below,  the  inputs  to  MSDF  that  are  readily  available  from  the  sensor  in 
its  current  version  are  listed  in  bold,  while  the  other  inputs  can  be  made  available  through 
further  processing,  either  in  an  improved  version  of  the  sensor,  or  in  the  data  fusion 
function  itself.  The  capability  of  post-flight  analysis  and  subsequent  update  of  the  MSDF 
databases  provided  by  the  FLIR  and  camera  exists  but  is  not  shown  in  the  table. 
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TABLE  VI 


Present  suite  of  sensors  and  their  potential  inputs  to  a  data  fusion  function 


Sensors 

Input  to  MSDF 

Accuracy 

(approximative) 

1.  Radar 

range 

0.45  m 

AN/APS-506 

bearing 

0.24° 

2.  ESM 

bearing 

*  The  accuracy  of  the  bearing  and  signal 

(ALR-502) 

emitter  ID 

characteristics  is  classified.  These  numbers  can  be 

threat  number, 

AOP 

platform  ID 

obtained  from  C-12-140-000/MB-z01  (p.4-36-1). 

3.  IFF 

range 

0.45  m 

AN/APX-502 

bearing 

0.24° 

allegiance 

N/A 

4.FLIR 

bearing 

*  Classified  C- 12-1 40-000/MB -zOl  (p.4-37-1) 

OR-5008/AA 

elevation 

* 

target  size  and  attitude 

N/A 

platform  ID 

N/A 

5.  Link-11 

Positions  (tracks  and 

*  Accuracy  is  limited  by  by  INS  and  GPS  see 

also  ID) 

[Ref.l],  p.I-54. 

6.ADP 

Position 

Doppler  together  with  bearing  and  time  delay 

OL/5004/AYS 

Velocity 

estimates  can  provide  very  accurate  position 

ID 

course  and  speed  estimates. 

7.  SRS 

Positions  of 

N/A 

AN/ARS-501 

up  to  31  sonobuoys 

ESSS9I 

Platform  ID 

N/A 

Target  position 

N/A 

In  Table  VII,  the  current  level  of  automation  for  each  sensor  is  described  along  with 
some  important  characteristics.  This  is  useful  when  estimating  the  level  of  effort  to  bring 
automatic  target  detection,  tracking  and  identification  (ATDTI)  onboard  the  aircraft. 
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TABLE  Vn 

Level  of  automation  of  the  present  suite  of  sensors 


Sensors 

Current  Level  of  Automation 

Characteristics 

1.  Radar 

AN/APS-506 

The  APS-506  is  currently  manually  j 
operated  except  for  the  capability  to 
automatically  track  one  target. 

No  digital  interface. 

X-band  pulse  compression  radar. 
PRF=500  Hz,  RPM=6.  Up  to  500 
messages/scan  (when  updated) 
Maximum  range:  150  nm 

2.  ESM 
AN/ALR-502 

The  ALR-502  currently  supplies 
digital  estimates  of  emitter  bearing 
and  associated  signal  characteristics. 

*  The  accuracy  of  the  bearing  and 
signal  characteristics  is  classified. 
These  numbers  can  be  obtained  from 
C-12-140-000/MB-z01  (p.4-36-1). 

3.  IFF 

AN/APX-502 

The  IFF  sensor  is  currently  totally 
unautomated.  The  IFF  cannot  process 
Mode  C  (altitude)  information. 

Unautomated,  cannot  process 
altitude  information. 

4.  FLIR 

OR-5008/AA 

Currently  the  FLIR  device  can  be 
manually  pointed  or  slaved  to  the 
radar  bearing.  There  is  no  computer 
automated  image  processing 
associated  with  the  FLIR.  The 
display  output  is  visually  interpreted 
by  the  system  operator. 

Useful  range  12-20  DM  (ships),  12- 
14  DM  (surface  subs),  6-8  DM  (sub. 
snorkel),  2-4  DM  (periscope). 

5.  Camera 
KA-501A 

No  automation 

(for  database  update),  lateral 
coverage  144° 

6.  MAD 

No  automation 

(vertically  below  flight  path) 

7.  Link-11 

The  Link- 1 1  interface  is  fully  digital 
and  is  automated. 

STANAG  5511  Compliant  Data  Link 
Information  Format 

8.  ADP 

OL/5004/AYS 

All  track  detection  and  update  is  done 
from  manual  inputs  from  the 
operator.  There  is  a  passive 
autodetect  algorithm  on  the  CP- 140 
although  it  is  not  used  much  because 
of  false  alarm  rates. 

The  ADP  processes  the  acoustic 
signals  to  determine  signal 
characteristics. 

9.  SRS 

AN/ARS-501 

The  SRS  is  interfaced  with  the 
computer  which  determines  the 
sonobuoy  positions  relative  to  the 
aircraft’s  geographic  position  and  the 
target  and  updates  buoy  positions  on 
the  Multi-Purpose  Display  (MPD) 
tactical  plots. 

Update  rate  is  one  sonobuoy  per 
second.  A  4-element  state  vector  and 
a  16-element  covariance  matrix  is 
maintained  for  each  sonobuoy 
through  a  Kalman  filtering 
procedure. 
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4.2  Review  of  the  Sensor  Upgrades  Currently  Anticipated  for  the  CP-140 

The  single  most  significant  addition  to  the  present  sensor  suite  is  a  Synthetic 
Aperture  (SA)  capability  that  could  be  added  to  the  AN/APS-506  search  radar  first 
upgraded  with  an  associated  digital  scan  converter  that  would  permit  automatic  processing 
of  the  radar  video  signal  in  order  to  provide  the  contact  input  data  to  the  fusion  function. 
The  addition  of  Electro-Optic  devices  (i.e.,  various  types  of  TV)  also  remains  a  possibility. 

Table  VIII  below  summarizes  the  additional  inputs  to  a  sensor  data  fusion  function 
provided  by  the  proposed  sensor  suite  upgrades  for  the  Aurora  and  their  intended  use  in 
addition  to  real-time  sensor  fusion.  To  this  list,  one  should  add  a  proposed  upgrade  to  the 
ESM  for  better  recognition  of  surface  targets  at  longer  range.  Before  any  advanced  data 
processing  (such  as  ATDTI)  is  done,  it  is  also  necessary  to  upgrade  the  Radar,  IFF  and 
FLDR.  with  digital  scan  converters  in  order  to  digitize  the  video  signal.  The  MSDF  inputs 
resulting  from  these  upgrades  would  however  be  the  same  as  in  Table  VI,  thereby 
justifying  their  exclusion  from  Table  VIII  to  prevent  redundancy. 

TABLE  VIII 

Proposed  suite  of  sensors  and  their  inputs  to  a  data  fusion  function 


Proposed  Sensors 

Input  to  Multi-Sensor  Data  Fusion  and  furt 
use 

SAR 

Range,  bearing,  platform  ID,  mapping,  target 
size,  video  recording  for  post-flight  analysis 

Updated  ADP  and  99-channel  SRS 

ADP  provides  processing  for  up  to  64  buoys 
SRS  provides  positioning 

Electro-Optics  (LLLTV,  Gated 
TV) 

Bearing,  elevation,  target  attitude,  mapping, 
video  recording  for  post-flight  analysis 

P228813.PDF  [Page:  45  of  90] 


UNCLASSIFIED 

33 


4.3  Review  of  the  Other  Information  Sources 

Table  IX  below  summarizes,  in  a  non-exhaustive  list,  other  information  sources  that 
can  be  used  for  data  fusion.  Although  Link-1 1  can  be  viewed  as  a  sensor  (Table  VI),  in  the 
sense  that  it  provides  already  processed  kinematic  and  identity  sensor  information  (tracks), 
it  can  also  be  viewed  as  a  source  of  non-sensor  information  data  with  an  associated  degree 
of  belief  to  be  ascertained  by  a  local  assessment  of  the  quality  and  timeliness  of  the  reports 
from  other  sources. 


TABLE  IX 


List  of  other  information  sources 


SOURCES 

Type  of  Information 

Link-11 

-  information  on  enemy  forces  (fixes,  etc.) 

-  information  on  own  forces  (PUs,  aircraft,  vessels, 
sonobuoys,  etc.),  and, 

-  information  of  tactical  importance  (splash  points, 
positions,  text  messages,  etc.). 

Visual  Sighting 

-  confirmation  by  operator  of  platform  ID  (e.g.,  after 
a  low  altitude  pass). 

Encyclopaedic  Data 

-  known  shipping  routes  and  air  corridors,  iceberg 
flow  patterns, 

-  special  events  that  can  lead  to  flotillas  of  similar 
ships, 

-  preferred  documented  submarine  routes. 

Operator  Input 

-  figure-of-merit  of  sensor  reports. 

Intelligence 

-  historical  database  of  manoeuvres  for  certain 
platform  types  or  ID. 

-  satellite  information  (e.g.,  tracking  of  targets  over 
several  days  by  foreign  government  agencies). 

-  prioritization  of  different  platform  databases  of 
anticipated  air  and  surface  targets. 

Environmental  data 

-  local  environment  along  the  flight  path. 
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Intelligence  reports  facilitate  the  identification  of  targets  by  allowing  the  fusion 
algorithm  to  look  up  a  smaller  portion  of  the  database.  The  fusion  function  should  allow 
for  environmental  data  to  affect  both  the  positional  uncertainty  in  the  reported  contact 
and/or  track  and  the  relative  quality  associated  with  a  given  information  source,  depending 
on  its  sensitivity  to  environmental  conditions.  It  is  reasonable  to  have  the  local 
environment  along  the  flight  path  set  the  priorities  and/or  weights  given  to  the  different 
source  reports. 

Some  operator  input  to  the  fusion  function  may  be  needed  depending  on  how  much 
automation  can  be  achieved  by  sensor  upgrades.  Apart  from  this  fact,  operators  are  the 
most  appropriate  judges  of  the  operational  state  of  all  sensors  on  board  and  should  be  able 
to  assign  a  figure-of-merit  to  each  sensor  and  provide  it  to  the  fusion  function.  During  in¬ 
flight  questioning  of  the  aircraft’s  crew,  this  particular  subject  was  mentioned  as  an 
invaluable  addition  that  commanding  officers  would  like  to  see  implemented  in  the  future. 


P228813.PDF  [Page:  47  of  90] 

UNCLASSIFIED 

35 

5.0  SENSOR  DATA  FUSION  PROCESS  FOR  THE  CP-140 

The  objective  of  Multi-Sensor  Data  Fusion  (MSDF)  is  to  enhance  the  ability  of  the 
tactical  crew  to  perform  their  missions.  In  carrying  out  the  previously  described  missions, 
the  Aurora  crew  is  bombarded  with  information  which  must  be  correlated,  fused  and 
interpreted  in  order  to  arrive  at  some  understanding  of  the  tactical  situation.  At  present, 
the  fusion  of  this  data  is  being  manually  performed  by  the  operators.  This  chapter  presents 
a  discussion  on  how  to  provide  automatic  target  tracking  and  identification  through 
MSDF  onboard  a  multi-sensor  platform  such  as  the  CP- 140. 

5.1  Top-Level  Functional  Decomposition  of  the  SDF  Process 

From  our  understanding  of  the  CP- 140  information  sources,  it  seems  advisable  to 
distinguish  three  distinct  entities  to  describe  a  top-level  functional  decomposition  of  the 
Sensor  Data  Fusion  (SDF)  process  as  shown  in  Figure  3: 

-  an  Under  Water  Sensor  Data  Fusion  (UWSDF)  centre  for  surface  and  subsurface 
target  tracking  and  identification; 

-  an  Above  Water  Sensor  Data  Fusion  (AWSDF)  centre  for  air  and  surface  target 
tracking  and  identification;  and 

-  a  common  track  database  that  insures  the  required  communication  between  the 
fusion  centres. 

The  wide  gap  between  the  requirements  that  each  fusion  centre  has  to  meet,  is 
mainly  responsible  for  this  type  of  architecture.  This  architecture  allows  different 
implementations  for  each  of  the  MSDF  functions  within  each  of  the  fusion  centres,  since 
the  problems  addressed  by  each  centre,  as  well  as  the  types  of  data  processed  and  the  time 
scales  involved,  are  quite  different.  Since  there  is  bound  to  be  some  overlap  in  the  targets 
for  which  each  centre  is  mainly  responsible,  a  common  database  insures  the  required 
communication  between  the  fusion  centres. 
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The  UWSDF  centre  mainly  deals  with  sonobuoy  Doppler  information  which  is 
subject  to  totally  different  clutter,  false  alarm  rates,  presence  of  ghosts,  environmental 
conditions  than  those  encounter  in  the  AWSDF  centre.  It  obviously  involves  different 
signal  processing  requirements  and  most  likely  different  fusion  algorithms.  The  state  of  the 
art  in  acoustic  data  processing  is  not  at  the  same  level  as  it  is  for  the  radar,  ESM  and  FLIR 
(AWSDF)  sensors. 

Automatic  detection  and  tracking  is  very  difficult  for  acoustics  sensors.  Given  the 
premise  that  surface  surveillance  should  be  the  focus  of  our  data  fusion  investigation,  no 
emphasis  will  be  put  on  the  UWSDF  centre  in  the  rest  of  this  report.  Our  sole 
recommendation  would  be  to  provide  tactical  decision  aids  to  assist  operators  to  output 
tracks  and  sometimes  ID  of  underwater  targets  and  large  surface  targets.  For  instance,  an 
automatic  acoustic  sensor  prediction  capability  has  been  suggested  as  a  very  useful 
decision  aids  (Ref.  12). 

Assuming  that  maritime  surveillance  is  the  most  frequent  operational  use  of  the  CP- 
140,  AWSDF  is  the  area  where  the  biggest  payoff  can  be  realized  by  the  automation  of  the 
tracking  and  identification  functions.  The  AWSDF  process  will  fuse  five  sensors  which  are 
the  radar,  the  IFF  sensor,  the  ESM  sensor,  the  FLIR  sensor,  and  the  Link  1 1  remote  track 
source.  Figure  4  shows  these  information  sources  that  interface  with  the  AWSDF 
process.  The  dotted  boxes  indicate  the  notional  upgrades  needed  to  provide  the  required 
information.  AWSDF  will  also  take  inputs  from  the  UWSDF  and  the  navigation  system. 
The  UWSDF  input  will  mainly  consist  of  tracks  for  targets  detected  by  sono-buoy  sensors. 
This  information  will  be  used  to  correlate  fused  surface  detections  from  radar,  ESM,  etc. 
with  acoustic  surface  tracks. 

The  top-level  functional  decomposition  of  Figure  3  allows  different  implementations 
for  each  of  the  MSDF  functions  within  the  AWSDF  centre.  Three  main  data  fusion 
functional  architectures  were  already  discussed  in  Chapter  2:  Sensor-Level  Fusion, 
Central-Level  Fusion,  and  Hybrid  Fusion.  Here,  we  show  an  appropriate  selection  of  these 
functional  architectures  to  achieve  target  tracking  and  identification  onboard  the  aircraft. 
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5.1.1  Recommended  MSDF  Architecture  for  Tracking 

For  the  tracking  aspect,  based  on  the  analysis  of  Refs.7  and  13  and  considering  the 
strengths  and  weaknesses  of  the  CP- 140  sensors,  DREV  is  recommending  a  hybrid  sensor 
fusion  architecture.  This  is  illustrated  on  Figure  5.  The  APS-506  radar,  the  IFF  sensor, 
and  the  OR-5008/AA  FLIR  all  have  good  resolution  and  accuracy  and  hence  are  ideal  for 
central-level  fusion.  The  ALR-502  ESM  sensor  has  very  poor  resolution  and  accuracy  in 
the  one  dimension  (bearing)  it  measures.  This  fact  makes  central-level  fusion 
inappropriate.  Therefore  the  sensor-level  fusion  approach  is  selected  in  this  case  in  order 
to  allow  an  ESM  track  to  first  develop  so  that  better  data  is  available  for  the  subsequent 
correlation  process.  The  remaining  sensors,  Link  1 1  and  acoustic  sensors  are  suitable  for 
sensor-level  fusion  since  their  outputs  are  already  in  track  form  and  not  in  contact  form. 

5.1.2  Recommended  MSDF  Architecture  for  Identification 

For  the  identification  aspect,  DREV  is  recommending  a  central-level  fusion 
architecture  to  ultimately  declare  an  ID  from  the  fusion  of  the  raw  target  features 
extracted  by  the  SAR  and  FLIR  sensors.  For  the  identity  information  provided  by  the 
other  sensors  (ESM,  IFF,  Acoustics)  or  by  non-organic  sources  (Link  1 1 )  already  offering 
identity  declarations  as  their  output  data,  the  sensor-level  architecture  is  an  appropriate 
approach.  This  architecture  (Fig.5)  can  be  adapted  to  include  identity  declarations  inferred 
by  operators  as  well  as  the  ones  deduced  by  non-organic  systems. 

5.1.3  The  MSDF  Track  Data  Base 

Finally,  the  track  database  contains  the  resulting  updates  of  the  kinematic  and 
identity  information  maintained  for  each  track  obtained  after  each  new  sensor  report  has 
been  fused.  A  list  of  probable  candidate  platform  IDs  should  be  continuously  updated, 
along  with  the  belief  in  that  ID.  In  addition,  some  other  information  about  the  track  should 
be  updated  (such  as  possible  allegiance,  nationality,  anticipated  threat  level,  etc.)  along 
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with  supports  of  these  assertions  (see  Chapter  2,  Table  I,  for  a  list  of  typical  track 
information  maintained  in  a  track  database). 

5.2  AWSDF  Tracking  and  Identification  Algorithms  for  the  CP-140 

In  this  section,  the  functions  constituting  the  AWSDF  process  and  necessary  to 
provide  automatic  target  tracking  and  identification  aboard  the  CP- 140  are  described, 
along  with  the  selected  appropriate  algorithms.  Figure  6  shows  a  detailed  functional 
architecture  capable  of  fusing  the  sensor  data  described  in  Figure  4.  The  processing  blocks 
are: 

•  data  alignment, 

•  data  association, 

•  target  state  estimation, 

•  target  identity  information  fusion, 

•  target  kinematic  behavior  assessment  (i.e.  type  of  maneuver,  etc.), 

•  track  fusion, 

•  track  management  process  (i.e.  initiation,  confirmation,  deletion,  etc.), 

•  cluster  management  process  (i.e.  merging,  splitting,  etc.), 

•  sensor  management, 

•  sensor  interface  and  control, 

•  track  database. 

The  data  flow  between  these  functions  is  shown  in  Figure  6.  For  some  functions,  a 
preferred  algorithm  has  emerged  as  a  result  of  our  trade-off  study. 
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FIGURE  3-  Top-level  functional  decomposition  of  the  Sensor  Data  Fusion  (SDF)  process 


RADAR 

(AN/APS-506) 


Energy 
Processing 
(Current  Senses) 


NAVIGATION  h 


AWSDF  Process 


Control  Data 


Sensor 

Management 


CP- 140  Tracks 


RADAR  I - 

(AN/APS-506)  ^Control 
- - -  Data 


Contacts 


IFF  ~~  7 

(AN/APX-502) 

_ _ _  Data 


FUR  - 

(OR-5008/AA)  ««ContTo1 
_ _  Data 


ESM  - 

(ALR-502)  ^Cowtro1 
- _ -  Data 


Contacts 


Contacts 


Contact  Data  Central-Level  „  Current 

**  Fusion  Tracks 


Track 
Database 
(Kinematic  &  ID) 


ESM 

Contacts  Data  Processing  Tracks 

►  •Single  Sensor  Tracker  ► 

•  Automatic  Target 
-  Tracking 


Track  Current  &  Updated 

Management  Tracks 


LINK  11 


Offboard  Track  Data 


Track  Data 


Onboard 
Track  Data 


Sensor-Level 

Fusion 


Current 

Tracks 


UWSDF 

PROCESS 


Surface 
Track  Data 


Surface  &  Subsurface  , 
Track  Data 


CP- 140  Tracks 


FIGURE  5  -  Top-level  sensor  fusion  architecture  of  the  AWSDF  process 
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FIGURE  6  -  Detailed  functional  decomposition  of  the  AWSDF  process 
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5.2.1  Sensor  Interface  and  Management 

An  interface  is  required  between  the  sensors  and  the  data  fusion  functions.  This 
interface  is  responsable  of  alignment  correction  and  sensor  post-processing  functions  for 
contact  quality  evaluation  and  reduction  of  false  alarms  before  the  sensor  data  is  sent  to 
the  fusion  process.  The  architecture  also  requires  a  sensor  management  function  to 
provide  feedback  communication  with  each  sensor.  In  general,  this  function  provides  such 
things  as  sensitivity  adjustments,  processing  control,  sensor  cueing,  ...etc. 

5.2.2  Data  Alignment 

This  process  must  perform  both  spatial  and  temporal  alignment.  Spatial  alignment 
performs  any  necessary  calculation  to  convert  the  contacts  and  tracks  to  the  same 
geopositional  frame  of  reference.  Spatial  alignment  must  take  into  account  all  possible 
sources  of  bias  which  could  corrupt  the  alignment  calculation.  These  sources  can  be  local 
to  the  Aurora  (vibration,  sensor  calibration  errors,  faulty  mounting,  false  true  north)  and 
they  can  also  originate  on  Participating  Units  (PUs)  whose  information  must  be  aligned 
with  the  CP- 140’s.  In  Figure  6,  the  spatial  alignment  is  shown  in  two  steps:  first,  the 
estimation  of  the  bias  in  the  sensor  alignment  box  and  second,  the  actual  correction  of  the 
incoming  contacts  or  tracks  in  each  of  the  sensor  interface  and  control  box. 

An  interesting  data  alignment  scheme  is  proposed  by  Norden  Systems  (Ref.7).  The 
first  step  in  the  alignment  process  is  to  identify  tracks  which  are  being  updated  by  multiple 
sensors,  have  high  track  quality  confidence,  and  are  not  maneuvering.  This  is  done  by 
looking  at  the  sensor  blip  scan  ratio,  the  track's  covariance  estimates,  the  track’s 
promotion  level,  and  the  time  since  last  update.  The  difference  between  the  input  contact 
or  track  and  the  reference  track  held  by  the  sensor  fusion  system  is  Calculated.  This  is 
done  by  time  aligning  the  sensor  fusion  track  to  the  contact  time  for  radar,  IFF,  and  ESM 
sensors  and  to  the  track  valid  time  for  the  acoustic.  Link  11,  and  ESM  tracks.  The 
difference  is  calculated  in  polar  (range,  bearing,  and  elevation)  coordinates.  The  biases  are 
accrued  using  a  low  pass  filter  so  that  noise  is  rejected  while  constant  biases  are  accurately 
estimated. 

The  APS-506  radar  is  used  as  the  reference  sensor.  The  other  sensors  use  the 
difference  between  their  bias  and  the  radar’s  bias  as  a  feedback  correction  to  the  input 
measurements.  The  alignment  system  operates  in  a  closed  loop  manner  until  all 
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uncorrected  biases  achieve  a  zero  mean.  When  alignment  is  enabled  by  the  system 
operator  a  monitor  function  detects  the  remaining  uncorrected  bias  and  then  turns  off  the 
alignment  function  when  the  remaining  uncorrected  bias  reaches  a  small  value.  If 
alignment  is  disabled  then  the  monitor  function  will  issue  an  operator  alert  if  the 
uncorrected  biases  reach  an  unacceptably  high  level.  The  bias  corrections  can  be  saved  in 
Electrically  Alterable  Read  Only  Memory  (EAROM)  for  use  in  future  mission  which  may 
not  have  targets  updated  by  multiple  sensors  (e.g.,  a  scenario  with  a  submerged  submarine 
and  a  surface  vessel). 

5.2.3  Data  Association 

The  function  of  the  data  association  process  is  to  determine  if  a  new  sensor  contact 
or  track  originates  from  an  existing  track,  or  requires  the  creation  of  a  new  track,  or  is 
simply  a  false  alarm.  Its  inputs  are  therefore  new  sensor  data,  and  existing  tracks  which  are 
time  updated  to  the  times  of  the  input  sensor  data.  In  general  data  association  can  proceed 
either  by  considering  data  generated  during  a  single  scan  or  during  multiple  scans.  In 
single  scan  mode,  nearest  neighbour  algorithms  (or  variants  thereof,  such  as  the  Jonger- 
Volgenant-Castanon  (JVC)  algorithm),  maximum  likelihood  estimators  and  the  Joint 
Probabilistic  Data  Association  (JPDA)  algorithms  can  be  used.  In  multiple  scan  mode,  one 
usually  employs  Multiple  Hypothesis  Tracking  (MHT)  algorithms  that  create  a  set  of 
statistical  hypotheses  that  must  be  tested  in  order  to  evaluate  if  the  input  data  correlates 
with  the  tracks.  The  number  of  hypotheses  must  be  maintained  of  manageable  proportion 
for  the  available  computing  power.  For  practical  computing  cost  reasons,  the  optimal 
algorithm  consisting  in  keeping  all  MHT  hypotheses  is  never  considered. 

Both  Refs. 7  and  13  recommended  the  MHT  as  an  appropriate  choice  for  the 
contact-to-track  association  process  (FLIR,  IFF,  Radar).  Because  of  the  large  target 
densities  expected  in  many  CP- 140  missions  and  the  possibility  of  imperfect  removal  of 
ocean  clutter,  the  association  method  should  cover  many  scans.  For  the  track-to-track 
association  process  (ESM,  LINK,  ACOUSTICS),  a  simpler  algorithm  such  as  a  nearest 
neighbour  (JVC)  would  suffice  because  the  tracks  are  well  established.  Ellipsoidal  gating 
is  recommended  before  applying  MHT  or  JVC  to  prevent  the  very  unlikely  association. 
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5.2.4  Target  State  Estimation 

This  process  mathematically  refines  the  state  of  motion  of  a  target  track,  by  fusing 
the  track’s  previous  state  vector  (position,  speed  and  covariance)  with  new  associated 
sensor  data.  In  some  cases,  the  target  behavior  assessment  process  could  suggest  a 
specific  model  for  target  dynamics  (constant  acceleration,  evasive  maneuvers  of  a  known 
type  for  the  target  ID).  Every  time  a  new  contact  arrives,  the  state  estimation  problem  can 
be  formulated  either  in  batch  mode  or  by  the  use  of  Kalman  filters  which  process  each  new 
contact  using  only  the  information  contained  in  the  track’s  state  vector  and  covariance 
matrix  at  the  last  contact  time.  Batch  methods  being  much  more  computer  intensive,  the 
usual  choice  is  to  perform  the  state  estimation  through  one  or  many  Kalman  filters  and  this 
is  the  preferred  method.  The  output  of  the  Kalman  filter  is  an  updated  state  vector  and 
covariance  matrix  at  the  time  of  the  fused  contact. 

5.2.5  Target  Kinematic  Behavior  Assessment 

This  process  supports  the  other  functions,  particularly  the  target  state  estimation,  by 
suggesting  target  behavior  models  based  on  target  characteristics  such  as  speed, 
acceleration  (previously  observed  or  potential),  direction  of  travel  (noting  potential 
encounters  with  geographical  features  or  other  targets)  and  manoeuvring  capability 
(possibly  by  looking  up  database  information). 


5.2.6  Target  Identity  Information  Fusion 

This  process  uses  fuzzy  notions  about  relative  physical  characteristics  of  targets  that 
can  restrict  the  list  of  possible  platforms  uniquely  satisfying  those  characteristics.  Radar 
cross-sections  (RCS),  temperature,  speeds  and  accelerations  are  characteristic  physical 
attributes  that  can  range  from  the  "Very  Large"  to  the  "Very  Small",  through  as  many 
increments  as  one  wishes,  with  flexible  boundaries  between  the  fuzzy  classes.  Attributes 
can  generically  be  described  as  a  fuzzy  characteristic  associated  to  a  platform  or  a  list  of 
platforms.  The  fuzzyness  can  be  the  result  of  measurement  error  on  a  physical  quantity,  or 
the  deliberate  binning  of  that  physical  quantity  into  distinct  finite  classes,  or  the  relative 
confidence  on  a  detectable  feature  of  the  platform. 
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An  estimate  of  speed  can  be  provided  by  the  target  state  estimation  and 
acceleration  can  be  obtained  through  the  kinematic  behavior  assessment  function.  Target 
attitude  can  be  assessed  from  the  FLIRand,  when  fused  with  radar  range  information,  can 
provide  estimates  of  side  and  forward  Radar  Cross-Sections  (RCS).  Temperature  of  a 
target  can  be  estimated  from  the  FLIR  image  contrast  relative  to  its  surroundings  (water 
or  air).  These  physical  characteristics  can  vary  in  time  through  either  target  behavior  or 
increased  accuracy  of  sensor  measurement.  For  the  purposes  of  determining  platform  ID, 
it  is  therefore  often  appropriate  to  provide  only  a  rough  (or  fuzzy)  classification  of  these 
physical  quantities,  such  as  Very  Big,  Big,  Medium,  Small  and  Very  Small.  To  each  class 
corresponds  a  list  of  known  platforms  possessing  the  given  characteristic. 

Some  sensors  suggest  possible  IDs  in  a  more  direct  way.  ESM  can  provide  plausible 
emitter  IDs  which  can  be  correlated  with  platform  IDs.  The  acoustic  signature  can  be 
processed  by  the  Acoustic  Data  Processor  (ADP)  and  provide  candidate  platform  IDs. 
Finally,  object  recognition  algorithms  applied  to  imaging  sensors,  such  as  the  SAR,  can 
provide  type  identification  (e.g.  tanker,  destroyer,  frigate,  etc.). 

Ref.  14  reviewed  a  representative  sample  of  identity  information  fusion  techniques: 
Bayesian  inference,  Dempster-Shafer  theory,  Altoft  reasoning  over  attributes,  and  fuzzy 
logic.  The  net  result  of  the  comparison  of  these  techniques  is  that  if  only  one  compromise 
method,  valid  for  all  missions,  is  to  be  chosen,  then  a  truncated  Dempster-Shafer 
algorithm  would  be  the  recommended  choice.  Ref.  14  provides  more  details  on  this 
technique. 


5.2.7  Track  Management  Process 

For  each  existing  track  that  is  updated  with  new  sensor  data,  the  results  of  the  State 
Estimation,  Target  Identity  Information  Fusion  and  Target  Kinematics  Behavior 
Assessment  functions  are  sent  to  the  Track  Management  (TM)  function  to  update  the 
track  database.  The  TM  function  must  also  initiate  a  new  track  for  each  unassociated 
contact,  confirm  a  track  after  several  contacts  are  associated  to  that  track  and  propose  as 
candidates  for  deletion  "bad"  tracks  for  which  no  contacts  have  been  reported  for  a  long 
time. 
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5.2.8  Cluster  Management  Process 

The  purpose  of  clustering  is  to  break  a  large  problem  into  a  set  of  smaller  ones 
which  are  solved  in  parallel.  A  cluster  is  a  set  of  tracks  with  common  measurements. 
Initially,  one  cluster  is  set  up  for  each  confirmed  target.  Each  new  measurement  is 
associated  with  a  cluster  if  it  falls  in  the  validation  region  of  any  track  from  that  cluster.  A 
new  cluster  is  set  up  for  each  unassociated  measurement.  Figure  7  shows  an  example  of 
the  clustering  mechanism.  If  a  measurement  is  associated  with  more  than  one  cluster  then 
those  clusters  are  merged  into  a  super  cluster  as  shown  on  Figure  8.  If  tracks  within  a 
cluster  separate  (Figure  9)  and  have  no  more  common  measurements,  then  this  cluster  is 
subdivided  accordingly  into  smaller  clusters.  The  process  of  initiating,  merging,  splitting 
and  deleting  clusters  is  referred  as  cluster  management. 

5.3  Hardware  Considerations 

Recent  trends  in  acquisitions  of  military  computer  hardware  have  shifted  from 
specialized  equipment  to  suitably  ruggedized  COTS  equipment.  Reduced  Instruction  Set 
Chips  (RISC)  running  the  UNIX  operating  system  are  a  particularly  popular  choice. 
Presently,  one  of  the  most  popular  data  bus  standard  is  the  VME  bus.  Norden  used  the 
Motorola  MVME  167  68040  based  single-board  computer.  Loral  implemented  the  Data 
Fusion  Demonstration  Model  (DFDM)  on  a  Sun  SparclO  Workstation. 
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FIGURE  7-  Clustering  mechanism 
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6.0  EXPECTED  PERFORMANCE  IMPROVEMENTS 

A  study  (Ref.  15)  has  been  conducted  to  assess  the  potential  for  enhancing  the  CP- 
140  crew  efficiency  by  incorporating,  among  other  technologies,  data  fusion.  The  data 
fusion  and  electronic  library  applications  were  found  to  offer  the  most  potential  to  enhance 
the  operator  efficiency.  This  was  attributed  to  the  increased  use  of  automation. 
Furthermore,  it  was  emphasized  that  these  technologies  (data  fusion,  electronics  libraries) 
might  have  a  significant  impact  on  overall  mission  effectiveness.  Our  intent  in  this  chapter 
is  not  to  investigate  crew  efficiency  or  crew  size  reduction  potential  which  are  beyond  the 
scope  of  this  report  but  to  briefly  summarize  the  potential  benefits  of  MSDF  in  terms  of 
operator  efficiency  and  mission  effectiveness. 

6.1  Impact  on  Operator  Efficiency 

From  a  workload  perspective,  data  fusion  would  generally  be  expected  to  result  in 
reduced  operator  workload  simply  from  the  automation  of  functions  that  would  otherwise 
be  performed  manually.  During  the  last  fifteen  years,  technological  advancements  have 
greatly  increased  sensor  performance  and  capability  to  generate  information.  As  a  result 
however,  the  operators  might  be  flooded  by  the  volume,  rate  and  complexity  of  the 
information  provided  by  these  sensors  to  a  point  where  their  ability  to  cope  with  that 
information  may  be  exceeded.  A  key  element  to  this  information  management  problem  is 
the  ability  to  automatically  combine  or  fuse  data  from  the  sensors. 

At  present,  the  fusion  of  this  data  is  being  manually  performed  by  the  operators 
through  intercom  communication,  with  little  support  from  the  Operational  Program 
beyond  that  provided  by  the  maintenance  of  the  tactical  database  and  the  ability  to  control 
the  various  sensors  “on-line”  through  the  use  of  a  common  Operator-Machine  Interface 
(OMI). 

At  the  lowest  level,  the  individual  sensor  operators  must  assimilate  the  information 
being  presented  to  them  from  their  sensor  and,  through  adjustment  of  the  sensor 
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operation,  they  must  resolve  ambiguities  in  order  to  reach  decisions  on  the  nature  and 
validity  of  the  information  being  presented.  For  example,  the  radar  operator  must  decide 
whether  a  particular  radar  return  is  a  target  or  merely  noise.  In  doing  so,  he  or  she  is 
manually  correlating  sensor  reports  with  known  information  on  the  mission  environment. 
Based  upon  the  results  of  the  decision,  the  operator  will  either  enter  a  contact  into  the 
tactical  database  or  discard  the  information  as  irrelevant. 

At  the  next  level,  the  Mission  Commander  must  integrate  the  inputs  from  the  aircraft 
sensors,  other  crew  members  and  friendly  forces  in  order  to  arrive  at  as  accurate  as 
possible  a  representation  of  the  tactical  situation.  Since  this  involves  the  manual 
correlation  of  tracks  and  additional  sensor  and  non-sensor  derived  inputs,  such  as 
intelligence  reports  and  communications  with  co-operating  forces,  assisted  by  the  use  of 
those  decision  aids  which  are  available  under  the  CP- 140  Operational  Program,  it  is  an 
extremely  demanding  task  which  requires  the  full  concentration  of  the  Mission 
Commander. 

Since  virtually  all  of  the  data  association  and  merging/combining  (constituting  the 
fusion  process)  which  occurs  during  the  course  of  an  Aurora  mission  is  presently 
performed  manually,  this  activity  contributes  a  significant  portion  of  the  overall  operator 
workload.  During  periods  of  high  activity,  the  operators  may  overlook  valid  information 
and  arrive  at  an  incomplete,  or  inaccurate,  assessment  of  the  tactical  situation.  As  well,  the 
varying  levels  of  skill  and  experience  among  the  operators  may  cause  different  individuals 
to  reach  different  conclusions  given  the  same  information. 

The  operators  function  in  the  upgraded  sensor  fusion  system  will  be  very  different 
from  what  the  operator  is  doing  in  the  presently  equipped  CP- 140  aircraft.  The  operator 
must  currently  manually  track  virtually  all  of  the  targets  which  the  aircraft  has  under 
surveillance.  This  is  a  very  time  consuming  process.  Experience  in  the  U.S.  Navy  has 
shown  that  a  good  operator  can  handle  no  more  than  six  manual  tracks  on  a  single  sensor 
at  any  one  time  (Ref.  7). 
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This  means  with  two  operators  doing  manual  tracking  a  maximum  of  12  tracks  can 
be  handled  by  the  aircraft  at  any  one  time  on  one  sensor.  If  sensor  fusion  is  desired  on  the 
targets  of  interest  then  the  operators  must  operate  more  sensors  and  hence  the  number  of 
tracks  or  the  track  quality  will  decline.  If  we  assume  that  the  two  operators  must  handle 
three  sensors  (radar,  IFF,  FLIR  or  ESM)  then  the  maximum  number  of  tracks  is  6 
tracks/operator  *  2  operators  /3  sensors  =  4  fully  fused  sensor  tracks. 

This  arithmetic  changes  drastically  when  an  automated  sensor  fusion  system  is 
implemented  on  the  aircraft.  The  number  of  tracks  which  can  be  simultaneously  tracked  is 
nearly  limitless  (greater  than  500).  The  operators  duties  now  become  one  of  track  and 
sensor  management.  The  operator  will  be  presented  with  a  clear  track  picture  with  all 
targets  held  by  the  sensor  fusion  system  being  clearly  displayed. 

The  operators’  job  will  be  to  insure  that  the  sensors  are  operating  properly  and  to 
make  judgment  calls  about  track  identity  and  intentions.  He  will  have  a  much  more 
complete  data  base  on  which  he  can  base  his  judgments. 

6.2  Impact  on  Mission  Effectiveness 

Table  X  below  summarizes  the  benefits  of  MSDF  which  impact  on  mission 
effectiveness.  Here  are  some  examples  of  how  those  MSDF  benefits  can  be  specifically 
related  to  CP- 140.  For  instance,  the  APS-506  radar  which  is  the  most  important  sensor  on 
the  aircraft  for  marine  surveillance  missions  will  allow  the  automated  surveillance  of  large 
areas  of  the  ocean  if  it  is  properly  equipped.  If  the  aircraft  is  at  an  altitude  of  25,000  feet 
an  area  of  about  79,000  square  miles  is  covered  by  the  radar.  At  an  altitude  of  10,000  feet 
an  area  of  32,000  square  miles  is  covered  with  antenna  in  horizon  sweep  mode.  These 
areas  are  far  too  large  for  manual  target  detection  and  tracking,  unless  the  radar  threshold 
is  set  high  enough  that  only  the  largest  and  closest  targets  are  displayed  for  manual 
tracking. 
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TABLE  X 

Potential  MSDF  Benefits  which  impact  on  Mission  Effectiveness 


MSDF  Benefits 

Explanations 

Robust  Operational 
Performance 

One  sensor  can  contribute  information  while  others 
are  unavailable,  denied  (jammed),  or  lack  coverage 
of  a  target/event 

Extended  Spatial  Coverage 

One  sensor  can  look  where  another  cannot 

Extended  Temporal 

Coverage 

One  sensor  can  detect/measure  a  target/event  when 
others  cannot. 

Increased  Confidence 

One  or  more  sensors  can  confirm  the  same 
target/event 

Reduced  Ambiguity 

Joint  information  from  multiple  sensors  reduces  the 
set  of  hypotheses  about  the  target/event. 

Improved  Detection 

Effective  integration  of  multiple  measurements  of 
the  target/event  increases  the  assurance  of  its 
detection. 

Enhanced  Spatial  Resolution 

Multiple  sensors  can  geometrically  form  a  synthetic 
aperture  capable  of  greater  resolution  than  a  single 
sensor  can  form. 

Improved  System  Reliability 

Multiple  sensor  suites  have  an  inherent  redundancy 

Increased  Dimensionality 

A  system  employing  different  sensors  to  measure 
various  portions  of  the  electromagnetic  spectrum  is 
less  vulnerable  to  disruption  by  enemy  action  or 
natural  phenomena. 

The  process  of  manually  tracking  a  target  is  also  very  time  consuming  and  error 
prone.  The  operator  must  first  discern  the  targets  video  on  the  PPI  or  B  scan  display.  He 
must  then  offset  the  radar  video  and  change  radar  scale  until  only  the  target  he  is  trying  to 
track  is  visible  on  the  display.  If  the  operator  keeps  the  radar  on  maximum  range  scale 
then  he  will  have  huge  tracking  errors.  If  a  12”  radar  PPI  display  is  used  and  the  radar  is 
in  a  150  mile  mode  then  an  error  of  .01  inches  corresponds  to  a  radar  measurement  error 
of  1,500  feet.  These  error  become  proportionately  worse  if  a  smaller  display  is  used. 
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It  is  impossible  to  determine  target  speed  or  heading  with  errors  of  the  magnitude 
stated  above.  Therefore  the  time  consuming  process  of  radar  offset  and  range  scale 
change  must  be  used  to  determine  the  target's  direction  and  speed.  While  the  operator  is 
updating  one  track  the  rest  of  the  surveillance  volume  is  unobserved  because  the  operator 
is  concentrating  only  on  the  track  being  updated  or  tracks  in  the  immediate  vicinity  of  the 
track  he  is  updating. 

Detection/Track  initiation  is  much  quicker  with  sensor  fusion.  One  reason  is  the 
mutual  support  provided  by  the  various  sensors  that  reduces  the  number  of  “holes”  in  the 
data  due  to  sensor  fades.  Secondly  the  aggregate  scan  period  of  the  suite  of  sensors  is 
significantly  less  than  the  fastest  sensor.  For  example  if  the  APS-506  radar  is  in  a  10 
second  scan  mode  and  the  FLIR  sensor  is  scanning  the  same  area  at  the  rate  of  once  every 
10  seconds  then  the  average  update  rate  for  a  target  being  seen  by  both  radar  and  the 
FLIR  is  once  every  5  seconds. 

Track  accuracy  is  better  because  tracks  are  updated  by  all  sensors  rather  than  having 
individual  tracks  on  each  sensor  and  selecting  the  "best"  track.  Better  track  continuity  is 
achieved  since  the  probability  that  all  sensors  are  experiencing  a  fade  at  the  same  point  is 
small.  The  overall  resolution  of  the  sensor  fusion  system  can  be  as  good  as  the  best 
sensor.  For  example  if  the  FLIR  has  a  bearing  resolution  of  0.01°  then  the  system 
resolution  will  approach  the  FLIR  resolution. 
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7.0  RECOMMENDED  IMPLEMENTATION  PLAN 

This  chapter  describes  the  way  DREV  sees  the  implementation  of  an  Automatic 
Target  Detection,  Tracking  and  Identification  (ATDTI)  capability  for  the  CP- 140.  A  large 
amount  of  hardware  and  software  required  to  realize  this  ATDTI  currently  exists  as 
Commercial-Off-The-Shelf  (COTS)  and  Govemment-Off-The-Shelf  (GOTS)  items.  Some 
modifications  to  the  existing  systems  will  have  to  be  made  to  tailor  them  for  the  CP- 140 
airborne  surveillance  mission. 

The  automation  can  be  achieved  in  a  number  of  different  ways,  and  depending  on  the 
level  and  the  sophistication  of  this  automation,  the  performance  gains  for  different 
missions  will  be  different.  It  is  important  to  identify  the  cost/performance  trade-offs  for 
implementing  increasing  levels  of  sophistication.  Also,  based  on  the  priorities  of  different 
types  of  missions,  the  implementation  of  the  automation  can  be  prioritised  to  start  from 
automating  the  highest  priority  missions  and  incrementally  add  automation  in  other 
missions  according  to  their  priority.  Currently,  maritime  surveillance  seems  to  be  the  most 
frequent  operational  use  of  the  aircraft. 

One  could  first  do  ATDTI  with  each  single  sensor  individually,  and  only 
subsequently  do  sensor  fusion.  However,  as  already  discussed  in  Chapter  2,  this  is  not  the 
optimal  way  of  doing  automated  surveillance  with  a  multiple  sensor  system  since  one 
limits  the  fusion  to  be  at  the  track  level  as  opposed  to  fusion  at  the  contact  level  which  is 
the  optimal  approach.  Moreover,  implementing  ATDTI  for  each  sensor  requires  to 
develop  capabilities  (such  as  track  association,  identification  and  track  management) 
which  are  required  anyway  for  fusion  both  at  the  track  level  and  at  the  contact  level. 
Hence,  the  most  efficient  design  would  be  to  develop  these  capabilities  to  achieve  optimal 
sensor  fusion  at  the  contact  level  when  appropriate. 

The  proposed  incremental  approach  shown  on  Fig.  10  is  a  way  to  implement  an 
ATDTI  capability  on  the  CP- 140  by  proceeding  with  recoverable  steps  where  different 
level  of  fusion  sophistication  can  be  progressively  implemented.  A  three  step  approach  is 
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put  forward  based  on  the  availability  of  the  technology  and  the  actual  status  of  the  sensors 
on  the  aircraft.  The  three  step  process  can  also  be  viewed  as  development  phases  yielding 
one  single  deliverable  system. 

7.1  STEP  Is  Baseline  ATDTI  with  Radar,  ESM,  Link  and  IFF 

Considering  the  current  status  of  the  sensors  on  the  CP- 140  (Chapter  4),  STEP  1 
represents  a  minimum  that  has  to  be  implemented  to  provide  an  efficient  ATDTI  capability 
aboard  the  aircraft.  This  is  an  optimal,  low  risk  implementation  that  would  require 
relatively  low  effort  since  most  of  the  required  hardware  and  software  currently  exists  as 
COTS  items.  Table  XI  below  gives  a  list  of  the  necessary  functions  to  realize  STEP  1 ,  as 
well  as  a  description  of  each  function  item  and  the  potential  modifications  required  on  the 
COTS/GOTS  items  that  could  eventually  be  used  for  its  implementation. 

In  this  initial  implementation  of  the  ATDTI  for  the  CP- 140,  the  radar  and  IFF 
sensors,  first  upgraded  with  off-the-shelf  cards  to  perform  Automatic  Target  Detection 
(ATD),  provide  contacts  to  the  Track  Management  System  (TMS)  developed  by 
Westhinghouse  Norden  Systems.  Fig.  11  illustrates  the  radar  and  IFF  initial  upgrades 
required  to  automatically  provide  contacts  to  the  TMS.  The  first  four  items  of  Table  XI 
describe  what  is  needed  to  implement  these  initial  sensor  upgrades. 
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FIGURE  10  -  Recoverable  steps  of  the  incremental  approach  recommended 
by  DREY  to  implement  sensor  data  fusion  on  the  CP- 140 


P228813.PDF  [Page:  70  of  90] 


UNCLASSIFIED 

58 


TABLE  XI 

STEP  1 :  Baseline  ATDTI  with  Radar.  ESM.  IFF  and  LINK 


Function 

Function  Item 

Description 

GOTS/COTS  Item 

Modifications 

Required 

Radar  Signal 
Digitization 

The  radar  Adaptive  Video 
Processor  (A VP)  converts 
APS-506  video  to  digital 
format. 

COTS  -  Norden  has  a 
shipbome  AVP  and  other 
companies  may  have 
some  as  well. 

Norden’ s  AVP  would 
need  modification  to 
operate  with  APS-506 
radar  parameters  and  in 
an  airborne 
environment. 

Radar  Automatic 
Target  Detection 

-  Thresholding  (CFAR) 

-  M-out-of-N  Detector 

-  Radar  Centroiding 

COTS  -  This  software 
exists  in  the  Norden 

AVP. 

The  software  would 
have  to  be  “tuned”  to 
the  APS-506  radar. 

IFF  Signal 
Digitization 

The  IFF  Beacon  Video 
Processor  (BVP)  converts 
IFF  video  to  digital 
format. 

COTS  -  Hazeltine, 
Telephonies. 

These  companies  have 
shipbome  IFF  BVP 
systems.  They  may 
have  airborne  IFF  BVP 
systems. 

IFF  Automatic 
Target  Detection 

The  IFF  Beacon  Video 
Processor  (BVP)  provides 
the  digital  contacts. 

COTS  -  Hazeltine, 
Telephonies. 

These  companies  have 
shipbome  IFF  BVP 
systems.  They  may 
have  airborne  IFF  BVP 
systems. 

Sensor  Interface 
and  Management 

-  Alignment  Correction 

-  Link  Gridlock 

-  Sensor  Activity  Monitoring 

-  False  alarm  reduction 

-  Contacts  Quality  Evaluatio 

-  Sensor  control 

COTS  -  This  software 
exists  in  the  Norden 

TMS. 

-Adaptive  Correlator 
-  Gridlock  function 
exists  for  the  NWS. 

The  software  currently 
only  handles  contact 
alignment.  The  track 
alignment  would  have  to 
be  added  for  the  LINK. 

Contact-to-Track 

Association 

-  This  function  is  needed 
for  radar  and  IFF  contacts. 

-  Gating 

-  Assignment 

COTS  -  This  software 
exists  in  the  Norden  TMS 

This  function  would 
require  little  or  no 
modification  for  the  CP- 
140  application. 

Target  State 
Estimation  & 
Kinematic 
Behavior 
Assessment 

-  These  functions  perform 
contact  fusion  and  state 
estimation 

COTS  -  This  software 
exists  in  the  Norden  TMS 

This  function  would 
require  little  or  no 
modification  for  the  CP- 
140  application. 
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TABLE  XI  ('Continued') 

STEP  1 :  Baseline  ATDTI  with  Radar.  ESM.  IFF  and  LINK 


Function 

Function  Item 

Description 

GOTS/COTS  Item 

Modifications 

Required 

Track 

Management 

-  This  function  performs 
tracks  initiation,  deletion 
and  confirmation. 

COTS  -  This  software 
exists  in  the  Norden  TMS 

This  function  would 
require  some  tuning  for 
the  CP- 140  application. 

Cluster 

Management 

-  This  function  performs 
cluster  initiation,  splitting 
and  merging. 

GOTS- 

DREV:  CASE_ATTT 
*  available  in  litterature 

This  function  has  to  be 
developed.  Not  expected 
to  be  a  big  effort. 

ESM  Tracking 

-  Automatic  tracking  for 
the  ESM 

COTS: 

Partially  developed  by 
Norden. 

An  ESM  tracker  has  to 

be  developed.  Most  of 
new  ESM  systems  have 
this  capability. 

Track-to-Track 

Association 

-Needed  for  the 
correlation  of  the  ESM  and 
data  link  tracks  with  the 
CP-140  composite  tracks. 

COTS: 

A  JVC  algorithm  exists 
at  Loral. 

The  JVC  algorithm  at 
Loral  must  be  verified 
for  the  CP- 140 
application. 

Track  Fusion 

-  Needed  for  the  fusion  of 
ESM,  LINK  and  CP- 140 
composite  tracks 

GOTS  - 

DREV:  CASE_ATTI 
*  available  in  litterature 

This  function  has  to  be 
developed.  Not  expected 
to  be  a  big  effort. 

Identity 

Information 

Fusion 

This  function  is  used  to 
automatically  declare  the 

ID  from  ESM,  IFF,  LINK 
and  kinematic  data. 

COTS-  This  software 
exists  in  the  Loral  DFDM 
demonstrator 

This  function  would 
require  some  tuning  for 
the  CP- 140  application. 

Sensor  Fusion 
System  Operator 
Display 

-  Needed  to  present  a 
coherent  picture  of  the 
entire  volume  of  space 
surrounding  the  CP- 140 

COTS  - 

Norden  has  a  display  for 
the  TMS 

Unknown 

Contacts 

•  Valid  Time 

•  Range 

•  Bearing 
•S/N 

•  AZ  Extent 

•  Range  Extent 

•  Track  Gate  No. 


Radar  Control 
Data 


Contacts 

•  Valid  Time 
•Range 

•  Bearing 

•  Allegiance 


IFF  Control 
Data 


for  step  1 
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The  TMS  will  achieve  automatic  tracking  using  both  radar  and  IFF  contacts.  This 
software  is  capable  of  performing  the  necessary  functions  (alignment,  association,  target 
state  estimation,  track  management,  etc.)  to  fuse  radar  and  IFF  contacts  and  generate 
tracks. 

The  ESM  sensor  currently  has  a  complete  digital  interface  which  could  be  used 
without  modification  to  provide  bearing  contacts  to  an  eventual  ESM  tracker  that  needs  to 
be  developed.  This  ESM  tracker  development  is  not  a  big  effort  since  most  of  the  modem 
ESM  systems  have  a  tracker.  Link  1 1  is  providing  its  data  in  track  form  and  the  major 
problem  with  using  this  data  is  to  properly  align  the  CP- 140’s  coordinate  system  to  the 
link  coordinate  system.  Gridlock  techniques  used  to  solve  this  problem  for  the  North- 
Warning  System  (NWS)  could  be  used  for  the  CP-140.  GPS  will  also  certainly  help  to 
solve  this  problem  asuming  the  other  platforms  use  GPS  as  well..  A  track-to-track 
association  algorithm  is  needed  for  the  correlation  of  the  ESM  and  Link-1 1  tracks  with  the 
CP- 140  composite  tracks.  Loral  has  developed  a  JVC  algorithm  for  the  DFDM  project 
(Ref.  16)  that  may  fulfill  this  requirement.  Finally  a  track  fusion  function  is  required  to 
fuse  ESM,  Link  and  CP- 140  composite  tracks. 

With  very  minimal  risk,  ESM  ID  information,  IFF  responses,  link  data  and  kinematic 
data  can  be  processed  by  a  truncated  DS  evidential  theory  algorithm  in  the  identity 
information  fusion  subprocess  of  AWSDF  (as  Loral  successfully  did  in  the  CPF  DFDM 
(Ref.  16)).  This  function  is  used  to  automatically  declare  the  target  ID  from  ESM,  IFF, 
LINK  and  kinematic  data. 

STEP  1  would  provide  automation  for  target  detection  and  tracking  as  well  as 
automatic  target  identification  in  most  of  the  surface  and  air  surveillance  and  patrol 
missions.  With  a  minimum  effort,  very  good  ATDTI  can  be  achieved  by  just  implementing 
this  step. 
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7.2  STEP  2:  Advanced  ATDTI  with  the  SAR  and  the  FLIR 

STEP  2  involves  medium  risk  technology  and  requires  more  effort  than  STEP  1. 
This  step  can  potentially  provide  great  benefits  for  both  tracking  and  identification  since 
the  radar  and  the  FLIR  are  the*  two  most  important  CP- 140  sensors  for  maritime 
surveillance.  The  fusion  of  radar  contacts  with  the  FLIR’s  digitized  imaging  data  will 
achieve  short  range  identification  while  the  capability  for  long  range  target  identification 
will  be  provided  by  the  SAR. 

Table  XII  lists  the  necessary  function  items  to  provide  the  advanced  ATDTI 
capability  on  the  CP- 140.  Figure  12  shows  what  modifications  are  required  on  the  actual 
radar  and  FLIR  to  provide  the  necessary  digital  information  to  the  AWSDF  system.  There 
are  digitizers  on  the  market,  referenced  as  frame  grabers,  that  convert  video  image  to 
digital  format.  A  SAR  mode  needs  to  be  fitted  to  the  CP-140  APS-506  radar.  A  contract 
has  been  recently  awarded  to  Loral  and  its  partners  to  achieve  that. 

Current  status  of  the  object  recognition  technology  and  real-time  constraints  dictate 
the  use  of  basic  image  processing  software  for  feature  extraction  in  order  to  keep  the  risk 
as  medium.  More  sophisticated  feature  extraction  techniques  that  are  still  considered  as 
laboratory  concepts  should  only  be  addressed  in  STEP  3.  The  actual  FLIR  on  the  CP- 140 
is  manually  operated  with  no  computer  automated  image  processing.  It  might  be  possible 
to  use  the  same  feature  extraction  software  for  both  the  SAR  and  the  FLIR. 

7.3  STEP  3:  Full  SDF  System  Implementation 

With  the  implementation  of  STEP  1  and  STEP  2,  the  Above  Water  Sensor  Data 
Fusion  (AWSDF)  centre  is  realized.  Assuming  that  the  role  of  the  aircraft  is  largely  for 
maritime  surveillance,  this  implementation  of  level- 1  data  fusion  with  the  above  water 
sensors  would  be  sufficient  to  provide  the  ATDTI  capability  to  the  CP- 140.  However, 
priorities  may  rapidly  change  and  this  may  necessitate  a  full  Sensor  Data  Fusion  (SDF) 
system  implementation  involving  acoustics  and  higher  levels  of  data  fusion  (i.e.,  levels  2, 
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3,  and  4  (see  Chapter  2)).  This  full  SDF  system  implementation  is  referred  to  as  STEP  3 
and  is  described  in  Table  XIH. 


TABLE  XII 

STEP  2:  Advanced  ATDTI  with  SAR  and  FLIR 


Function 

Function  Item 

Description 

GOTS/COTS  Item 

Modifications  Required 

STEP  1 

see  STEP  1  Table 

Parts  of  the  software 
may  require  some 
modifications  with  the 
addition  of  SAR  and 
FLIR. 

SAR  Upgrade 

-  Automatic  Target 
Detection  and  Features 
Extraction 

COTS  &  GOTS: 

Loral  &  DREO  have  the 
necessary  software  & 
hardware 

N/A 

FLIR  Video 
Processor 

The  FLIR  Video 

Processor  converts  FLIR 
video  into  digital  format. 

COTS: 

Frame  Graber 

Unknown  but  expected  to 
be  little. 

FLIR  Upgrade 

-  Automatic  Target 
Detection 

-  Thresholding,  centroiding 

-  Automatic  Features 
Extraction 

COTS: 

Basic  Image  Processing 
Software 

Verify  if  the  software 
developed  for  the  SAR 
might  be  used  for  the 
FLIR  (Feature 
extraction) 

Identity 

Information 

Fusion 

-  Needed  for  the  fusion  of 
identity  information  from 
SAR,  FLIR,  IFF,  ESM, 
LINK  and  kinematic  data. 

COTS-  This  software 
exists  in  the  Loral 

DFDM  demonstrator 

This  function  would 
require  some 
modifications  to  accept 
features  from  FLIR  and 
SAR 

Sensor  Fusion 
System 
Operator 
Display 

-  Needed  to  present  a 
coherent  picture  of  the 
entire  volume  of  space 
surrounding  the  CP-140 

COTS- 

Norden  has  a  display 
for  the  TMS 

Unknown 

STEP  3  involves  high  risk  technology  and  could  require  extensive  effort.  To  mitigate 
the  risk,  a  proof-of-concept  phase  will  be  needed  as  well  as  further  research  in  some  areas 
where  very  sophisticated  algorithms  have  to  be  implemented.  We  have  chosen  to  separate 
this  step  from  the  first  two  since  some  technology  issues  still  have  to  mature. 


RADAR 

(AN/APS-506) 


Energy 
Processing 
(Current  Sensor) 


Contacts 

•  Valid  Time 

•  Target  Type 

•  Target  Size 

•  Aspect  Angle 

•  Heading 


Radar  Control 
Data 


Contacts 

•  Valid  Time 

•  Bearing 

•  Elevation 

•  AZ  Extent 

•  EL  Extent 

•  Target  Type 

•  Target  Size 

•  Aspect  Angle 

•  Heading 

FLIR  Control 
Data 


2 
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Currently,  feature  extraction  techniques  for  real-time  application  mostly  rely  on  very 
basic  image  processing  software.  More  sophisticated  feature  extraction  and  object 
recognition  techniques  have  to  be  developed.  In  this  respect,  the  ongoing  Defence 
Industrial  Research  (DIR)  project  entitled  “Context-Sensitive  Data  Fusion’’  (Ref.  17) 
awarded  to  MacDonald  Dettwiler  will  certainly  deliver  useful  tools  for  features  extraction 
in  terms  of  image  fusion  algorithms,  visualization  techniques  and  merging  of  imagery  with 
tactical  information.  Another  relevant  ongoing  activity,  is  the  Loral/Queens  NSERC  grant 
project  on  the  fusion  of  information  from  imaging  and  non-imaging  sensors.  This  project 
will  develop  MSDF  algorithms  for  distant  airborne  detection,  tracking  and  recognition  of 
objects  for  ocean  surveillance. 

The  automatic  analysis  of  the  acoustic  signal  is  another  extremely  challenging 
problem  often  involving  numeric,  AI  and  neural  networks  approaches.  These  approaches 
process  and  fuse  multiple  signals  from  multiple  sensors  to  derive  passive  bearings,  fixes, 
tracks  and  platform  attribute  data.  The  data  on  surface  and  subsurface  targets,  available  to 
the  automatic  sonar  processors  through  the  common  MSDF  database,  will  help  decide  on 
the  sonobuoy  deployment  pattern,  and  will  help  localise  the  target  much  sooner. 

The  availability  of  COTS  and  GOTS  components  for  the  upgrade  of  the  acoustic 
system  is  somewhat  more  limited  than  for  the  non-acoustic  sensors.  DREA  is  working  on 
an  experimental  Acoustic  Sensor  Prediction  Capability.  Currently,  Loral  (US)  has  an  R&D 
project  supported  by  DoD  to  study  various  techniques  of  acoustic  signal  processing 
(decluttering,  feature  extraction)  which  could  be  applied  in  the  CP- 140  case.  Westinghouse 
Norden  systems  is  currently  doing  work  for  the  U.S.  Navy  under  the  WLY-1  program. 
Acoustic  data  has  been  digitally  recorded  during  actual  sea  trials  and  playback  into  a  prototype 
system  that  performs  automatic  detection  and  tracking  functions.  This  prototype  system  has  the 
demonstrated  capability  to  maintain  tracks  on  over  1000  frequency  lines  and  has  the  storage 
capability  to  maintain  tracks  on  up  to  8000  lines  simultaneously. 

The  implementation  of  higher-level  data  fusion  for  both  the  AWSDF  and  the 
UWSDF  requires  substantial  research  effort  prior  to  proceeding  to  any  implementation. 
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No  COTS  or  GOTS  items  are  really  available  to  achieve  these  levels  of  fusion,  except 
perhaps  the  decision  aid  tools  that  are  currently  being  developed  under  the  MDA  DIRP 
and  a  NSERC  project  on  adaptive  planning  with  Loral/DREV  and  the  University  of 
Montreal. 

TABLE  XIII 

STEP  3:  Full  SDF  System  Implementation 


Function 

Function  Item  Description 

GOTS/COTS  Item 

STEP  2 
(refinements) 

see  STEP  2  Table 

-  More  sophisticated  feature 
extraction  techniques 

-  Image  fusion  (SAR  &  FLIR) 

-  MDA  DIRP  on  Context- 
Sensitive  Data  Fusion 

-  Loral/Queens  NSERC 
project 

Acoustic 

ATDTI 

-  This  function  will  provide 
acoustic  track  position,  velocity 
accuracy  and  classification 
data  from  sunoboys. 

GOTS  &  COTS  - 
Software  exists  at  Loral  (US), 
Westinghouse  (US). 

UWSDF 

-  Tactical  Decision  Aids 
(Automatic  Acoustic  Sensor 
Prediction  Capability  (ASPC)) 

-  Acoustics  Sensor  Fusion 

GOTS- 

DREA  is  working  on  tactical 
decision  aids  for  integrated 
active/passive  processing 

Level-2-3 

Data  Fusion 

-  Situation  Assessment  (time, 
space)  and  Threat  Assessment 

-  Pattern  recognition  (e.g., 
flotilla  deployment).. 

COTS: 

MDA  DIRP 

Level-4  Data 
Fusion 

-  Resource  and  Sensor 
management:  planning  and 
decision  support  functions  to 
allocate,  schedule  and  select 
the  best  course  of  action  in 
support  of  the  mission. 

GOTS: 

DREV  has  some  algorithms 
for  adaptive  planning. 

Loral  is  starting  some  work  on 
that. 

Advanced 

HCI 

-  Needed  to  present  full  data 
fusion  capabilities. 

Unknown 

Adjustments 
might  be 
required. 


-  Unknown 


A  demonstration 
system  has  been 
flown  in  an 
Aurora 


-  Proof-of- 
concept is 
required. 


-  Proof-of- 
concept  is 
required. 
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7.4  Final  Considerations  on  the  Recommended  Implementation  Plan 

The  purpose  of  the  implementation  plan  recommended  in  this  chapter  was  to  clarify 
what  is  currently  available  and  ready  to  be  implemented  at  low  cost  and  risk,  what  could 
be  implemented  at  medium  cost  and  finally  what  is  very  risky  and  needs  substantial  effort. 
This  was  the  motivation  underlying  the  three  step  approach  put  forward.  In  fact,  it  is 
premature  at  this  moment  to  undertake  an  implementation  phase,  to  acquire  or 
manufacture  the  system  described  in  this  report.  We  strongly  suggest  to  first  conduct  a 
low  cost  Project  Definition  Study  (PDS)  sub-phase.  The  purpose  of  this  PDS  sub-phase 
would  be  to  refine  the  approach  proposed  in  this  report  by  providing  its  complete  costing 
and  a  better  assessment  of  the  suggested  implementation  steps  in  terms  of  time,  risk  and 
complexity.  This  would  provide  to  ALEP  the  basis  for  the  SOW  for  the  more  costly 
implementation  phase. 

The  PDS  sub-phase  shall  be  partionned  into  the  following  tasks: 

1)  review  the  DREV/NORDEN/LORAL  documents; 

2)  analyse  and  refine  the  steps  proposed  by  DREV; 

3)  detail  the  tracking  and  identification  processes  needed  to  implement  STEP1; 

•  identify  the  refinements  required  for  the  already  developed  algorithms; 

•  identify  the  algorithms  that  needs  to  be  developed; 

•  maximize  the  use  of  COTS/GOTS  and  identify  the  missing  parts; 

•  identify  the  HCI  requirements; 

•  provide  a  design  diagram  to  implement  STEP  1 ; 

•  evaluate  the  risk  and  cost  the  efforts. 

4)  detail  the  processes  to  implement  STEP2  and  STEP3; 

•  identify  the  refinements  required  for  the  already  developed  algorithms; 
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•  identify  the  algorithms  that  needs  to  be  developed; 

•  maximize  the  use  of  COTS/GOTS  and  identify  the  missing  parts; 

•  identify  the  HCI  requirements; 

•  evaluate  the  risk  and  cost  the  efforts; 

5)  conduct  a  trade-off  study  between  the  steps  in  terms  of  efforts  and  expected  benefits; 

6)  evaluate  the  first  prototype  development  cost  for  each  step  individually; 

7)  evaluate  the  subsequent  implementation  cost  per  aircraft  for  the  each  step  individually. 
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8.0  CONCLUSION 

The  purpose  of  the  project  discussed  in  this  document  was  to: 

1 .  study  the  feasibility  of  implementing  data  fusion  aboard  CP- 140; 

2.  clarify  the  terminology  and  specify  the  data  fusion  capabilities; 

3.  based  on  the  Canadian  Patrol  Frigate  (CPF)  expertise  built  up  at  Loral, 
examine  what  could  be  transported  to  the  CP- 140  aircraft; 

4.  based  on  Westinghouse  Norden  Systems  fielded  data  fusion  technology  to 
the  USN  ships  and  their  proposal  to  the  P3  aircraft,  examine  what  could  be 
transported  to  the  CP- 140  aircraft; 

5.  develop  sensible  requirements  for  the  CP- 140  aircraft  based  on  the  results  of 
this  task;  and 

6.  ensure  the  feasibility  of  the  requirements  imposed  on  the  CP- 140  aircraft. 

The  scope  of  the  study  was  imposed  to  maritime  surveillance  with  the  non-acoustic 
sensors.  The  acoustics  perspective  was  seen  too  demanding.  This  limitation  in  scope  was 
not  to  re-priori tize  the  CP- 140  missions  nor  to  investigate  crew  efficiency  or  crew  size 
reduction  potential.  It  provided  better  assessment  of  the  benefits  of  data  fusion  against 
those  missions. 

This  report  presented  the  final  results  on  the  feasibility  and  usefulness  of  data  fusion 
for  the  CP- 140  Aurora  maritime  patrol  aircraft.  Relevant  sensor  fusion  concepts  and 
terminology  have  been  defined  along  with  a  description  of  the  CP- 140  operational 
environment  and  information  sources.  An  analysis  of  applicable  sensor  fusion  processes 
has  been  presented  followed  by  a  discussion  on  the  expected  performance  improvements. 
Finally,  a  three  step  incremental  approach  has  been  proposed  with  recoverable  steps  where 
different  level  of  fusion  sophistication  can  be  implemented  based  on  the  availability  of  the 
technology  and  the  actual  status  of  the  sensors  on  the  aircraft.  The  three  step  process  can 
be  viewed  as  development  phases  yielding  one  single  deliverable  system. 
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It  was  judged  premature  at  this  moment  to  undertake  an  implementation  phase,  to 
acquire  or  manufacture  the  system  described  in  this  report.  It  was  suggested  to  first 
conduct  a  low  cost  Project  Definition  Study  (PDS)  sub-phase.  The  purpose  of  this  PDS 
sub-phase  would  be  to  refine  the  approach  proposed  in  this  report  by  providing  its 
complete  costing  and  a  better  assessment  of  the  suggested  implementation  steps  in  terms 
of  time,  risk  and  complexity.  This  would  provide  to  ALEP  the  basis  for  the  SOW  for  the 
more  costly  implementation  phase. 

The  results  presented  in  this  report  will  be  used  to  derive  reasonable  and  prioritized 
requirements  for  the  CP- 140.  The  report  will  be  provided  as  reference  documentation  to 
the  ALEP  definition  contractor  as  supporting  data.  The  definition  contractor  will  then 
make  its  proposal  for  implementation. 
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